02 - Privacy-First Setup Guide

02 - Privacy-First Setup Guide

🔒 100% Local Mode - Zero Cloud Dependencies
⏱️ Time Estimate: 20-30 minutes
📋 What You’ll Learn: How to set up Selfoss with complete local processing using Ollama and Whisper.cpp

---

Table of Contents

• Why Choose 100% Local Mode?
• Prerequisites
• Installing Ollama
• Downloading Whisper.cpp Models
• Configuring Selfoss
• Testing Your Setup
• Understanding Model Trade-offs
• Disk Space Requirements
• Troubleshooting

---

Why Choose 100% Local Mode?

✅ Benefits

Feature	Local Mode	Cloud Mode
Privacy	🔒 Complete - data never leaves your device	⚠️ Data sent to AI providers
Cost	✅ Free forever (after initial setup)	💰 Pay-per-use API costs
Internet	✅ Works offline	❌ Requires connection
Data Retention	✅ No external storage	⚠️ Stored by providers temporarily
Speed	⏱️ Moderate (depends on hardware)	⚡ Fast (cloud GPUs)
Setup	🔧 Requires installation	✨ Just need API keys

🎯 Perfect For:

• 💼 Confidential business meetings
• 🏥 Healthcare discussions (HIPAA compliance)
• 💰 Financial planning sessions
• 🔐 Security-sensitive environments
• 🌍 Offline or low-connectivity scenarios
• 📉 Cost-conscious users with high volume

---

Prerequisites

Hardware Requirements

Minimum:

• CPU: Quad-core processor (Intel i5 or AMD equivalent)
• RAM: 8GB (16GB recommended)
• Storage: 10GB free space for models
• GPU: Optional (speeds up processing significantly)

Recommended for Best Performance:

• CPU: 8+ cores
• RAM: 16GB+
• GPU: NVIDIA GPU with 6GB+ VRAM (for GPU acceleration)
• Storage: SSD with 20GB+ free space

💡 Pro Tip: GPU acceleration can reduce transcription time by 5-10x. If you have an NVIDIA GPU, make sure to install CUDA drivers.

---

Installing Ollama

Ollama provides the local LLM infrastructure for text analysis.

Windows

1. Download Ollama:

   • Visit https://ollama.com/download
   • Download the Windows installer

2. Install:

   # Run the downloaded installer
   # Follow the on-screen instructions

3. Verify Installation:

   ollama --version
   # Should output: ollama version x.x.x

4. Pull Required Models:

   # For text analysis (required)
   ollama pull llama3.1:latest
   
   # For transcription (required)
   ollama pull whisper:base
   
   # Optional: Larger models for better accuracy
   ollama pull llama3.1:70b
   ollama pull whisper:large

⏱️ Download Time: 5-15 minutes per model (depending on internet speed)

macOS

1. Download Ollama:

   # Visit https://ollama.com/download
   # Download the macOS installer (.dmg)

2. Install:

   • Open the .dmg file
   • Drag Ollama to Applications
   • Launch Ollama from Applications

3. Verify Installation:

   ollama --version

4. Pull Required Models:

   ollama pull llama3.1:latest
   ollama pull whisper:base

Linux

1. Install via Script:

   curl -fsSL https://ollama.com/install.sh | sh

2. Verify Installation:

   ollama --version

3. Start Ollama Service:

   # Start as service
   sudo systemctl start ollama
   
   # Enable on boot
   sudo systemctl enable ollama

4. Pull Required Models:

   ollama pull llama3.1:latest
   ollama pull whisper:base

Testing Ollama

# Test text generation
ollama run llama3.1:latest "Hello, how are you?"

# Test transcription (requires audio file)
# Ollama will be tested via Selfoss interface

✅ Success: If you see a response, Ollama is working!

---

Downloading Whisper.cpp Models

Whisper.cpp provides local audio transcription without external APIs.

Understanding Whisper Models

Model	Size	Speed	Accuracy	Best For
tiny.en	~75MB	⚡⚡⚡ Very Fast	⭐⭐ Basic	Quick notes, clear audio
base.en	~140MB	⚡⚡ Fast	⭐⭐⭐ Good	General meetings, standard quality
small.en	~460MB	⚡ Moderate	⭐⭐⭐⭐ Very Good	Professional meetings, important content
medium.en	~1.5GB	🐢 Slow	⭐⭐⭐⭐⭐ Excellent	High-accuracy needs, technical content
large-v3	~3GB	🐢🐢 Very Slow	⭐⭐⭐⭐⭐ Best	Critical transcripts, noisy audio

💡 Recommendation: Start with base.en for the best balance of speed and accuracy.

Installing Whisper.cpp (via Ollama)

The easiest way is to use Ollama’s Whisper integration:

# Download recommended model
ollama pull whisper:base

# Optional: Download other sizes
ollama pull whisper:tiny      # Fastest
ollama pull whisper:small     # Better accuracy
ollama pull whisper:medium    # High accuracy
ollama pull whisper:large     # Best accuracy

Manual Whisper.cpp Installation (Advanced)

If you want to use Whisper.cpp directly (without Ollama):

Windows/macOS/Linux:

1. Download models from Hugging Face
2. Place in: ~/.cache/whisper/ (Linux/macOS) or %USERPROFILE%\.cache\whisper\ (Windows)

---

Configuring Selfoss

Now configure Selfoss to use your local setup.

Step 1: Open Settings

1. Launch Selfoss
2. Click ⚙️ Settings in the header
3. Navigate to LLM & Processing section

Step 2: Configure Transcription Provider

For Audio → Text (Transcription):

1. Provider: Select Ollama
2. Model: Select whisper:base (or your chosen model)
3. Ollama Endpoint: Leave as http://localhost:11434 (default)
4. API Key: Leave empty (not needed for local)

┌─────────────────────────────────────┐
│ Transcription LLM Settings          │
├─────────────────────────────────────┤
│ Provider:     [Ollama           ▼]  │
│ Model:        [whisper:base     ▼]  │
│ Endpoint:     http://localhost:11434│
│ API Key:      (leave empty)         │
└─────────────────────────────────────┘

Step 3: Configure Analysis Provider

For Text → Insights (Analysis):

1. Provider: Select Ollama
2. Model: Select llama3.1:latest
3. Ollama Endpoint: Leave as http://localhost:11434
4. API Key: Leave empty

┌─────────────────────────────────────┐
│ Analysis LLM Settings               │
├─────────────────────────────────────┤
│ Provider:     [Ollama           ▼]  │
│ Model:        [llama3.1:latest  ▼]  │
│ Endpoint:     http://localhost:11434│
│ API Key:      (leave empty)         │
└─────────────────────────────────────┘

Step 4: Enable Automation (Optional)

Toggle these settings for convenience:

• ✅ Auto-transcribe after recording: Automatically process audio
• ✅ Auto-analyze after transcription: Automatically generate insights

⚠️ Note: Auto-analyze will start immediately after transcription completes.

Step 5: Save Settings

Click “Save Settings” at the bottom of the page.

✅ Success: You’ll see a confirmation toast notification.

---

Testing Your Setup

Test 1: Check Ollama Connection

1. Go to Settings → LLM & Processing
2. Verify you see your models listed
3. Endpoint should show http://localhost:11434

✅ Success: Models are loaded and ready.

Test 2: Record and Transcribe

1. Create a test project
2. Click the microphone icon 🎤
3. Record a short message (10-15 seconds)
4. Stop recording
5. Wait for auto-transcription (or click “Transcribe Audio”)

⏱️ Expected Time:

• Whisper tiny: ~5-10 seconds
• Whisper base: ~15-30 seconds
• Whisper small: ~30-60 seconds

✅ Success: You see transcribed text in the transcript view.

Test 3: Analyze Transcript

1. Upload a test transcript file (.txt)
2. Click “Start Analysis”
3. Wait for processing

⏱️ Expected Time:

• Short transcript (1 page): ~10-20 seconds
• Medium transcript (5 pages): ~30-60 seconds
• Long transcript (20+ pages): 2-5 minutes

✅ Success: You see decisions, actions, and concepts visualized.

---

Understanding Model Trade-offs

Transcription Models

Whisper Tiny (75MB)

• ✅ Fastest processing (real-time capable)
• ✅ Minimal disk space
• ❌ May miss technical terms
• ❌ Struggles with accents
• Use for: Quick voice notes, clear audio

Whisper Base (140MB) ⭐ Recommended

• ✅ Good balance of speed and accuracy
• ✅ Handles most accents well
• ✅ Reasonable disk space
• Use for: General meetings, standard transcription

Whisper Small (460MB)

• ✅ Excellent accuracy
• ✅ Better with technical terminology
• ❌ Slower processing (3-4x base)
• Use for: Important meetings, professional content

Whisper Large (3GB)

• ✅ Best possible accuracy
• ✅ Handles noisy audio well
• ❌ Very slow (10x base)
• ❌ Large disk space required
• Use for: Critical transcripts only

Analysis Models

Llama 3.1 (4GB)

• ✅ Good general-purpose model
• ✅ Fast inference
• ✅ Handles most business content
• Use for: Standard meeting analysis

Llama 3.1 70B (40GB)

• ✅ State-of-the-art accuracy
• ✅ Better reasoning
• ❌ Requires 48GB+ RAM
• ❌ Much slower processing
• Use for: Complex strategic discussions

💡 Pro Tip: Use base for transcription and llama3.1 for analysis. This gives you the best performance/quality balance for most use cases.

---

Disk Space Requirements

Base Setup (Minimum)

Ollama:              ~500MB
whisper:base:        ~140MB
llama3.1:latest:     ~4GB
─────────────────────────────
Total:               ~5GB

Recommended Setup

Ollama:              ~500MB
whisper:base:        ~140MB
whisper:small:       ~460MB
llama3.1:latest:     ~4GB
─────────────────────────────
Total:               ~5.5GB

Full Setup (All Models)

Ollama:              ~500MB
whisper:tiny:        ~75MB
whisper:base:        ~140MB
whisper:small:       ~460MB
whisper:medium:      ~1.5GB
llama3.1:latest:     ~4GB
llama3.1:70b:        ~40GB
─────────────────────────────
Total:               ~47GB

Audio Recording Storage

Recordings are stored in:

• Windows: C:\Users\{Username}\AppData\Roaming\selfoss\audio_recordings\
• macOS: ~/Library/Application Support/selfoss/audio_recordings/
• Linux: ~/.local/share/selfoss/audio_recordings/

Estimate: ~1MB per minute of audio (WebM format)

• 1 hour meeting: ~60MB
• 10 hours: ~600MB
• 100 hours: ~6GB

💡 Pro Tip: Set up periodic cleanup of old recordings to save space. See 09_DATA_MANAGEMENT_GUIDE.md.

---

Troubleshooting

Ollama Issues

“Cannot connect to Ollama” error

# Check if Ollama is running
ollama list

# Restart Ollama (Windows)
# Close from system tray and relaunch

# Restart Ollama (Linux)
sudo systemctl restart ollama

# Check endpoint
curl http://localhost:11434/api/version

Models not appearing in Selfoss

# List installed models
ollama list

# Pull missing models
ollama pull whisper:base
ollama pull llama3.1:latest

Slow transcription on CPU

• ✅ Close other applications to free RAM
• ✅ Use smaller model (tiny or base)
• ✅ Consider GPU acceleration

Whisper.cpp Issues

“Model not found” error

• ✅ Verify model is downloaded: ollama list
• ✅ Re-download: ollama pull whisper:base
• ✅ Restart Ollama service

Empty transcription results

• ✅ Check audio file size (must be > 1KB)
• ✅ Verify audio duration (minimum 1 second)
• ✅ Test with a longer recording (30+ seconds)
• ✅ Try a different model

Very slow processing

• ✅ Use a smaller model (tiny or base)
• ✅ Close other applications
• ✅ Check CPU usage in task manager
• ✅ Consider upgrading hardware

Performance Optimization

Speed up transcription:

1. Use whisper:tiny for quick drafts
2. Enable GPU acceleration (NVIDIA GPUs only)
3. Close resource-intensive applications
4. Upgrade RAM if using large models

Speed up analysis:

1. Use standard llama3.1:latest (not 70B)
2. Process shorter transcripts
3. Disable auto-analyze for batch processing
4. Consider cloud provider for complex analysis

---

Next Steps

🎉 Congratulations! You’ve set up 100% local processing.

Recommended Actions:

1. 📊 Test with real meetings - Record or upload actual transcripts
2. ⚡ Optimize models - Experiment with different sizes for your hardware
3. 💾 Set up backups → 09_DATA_MANAGEMENT_GUIDE.md
4. 🔄 Try hybrid mode - Use local transcription + cloud analysis → 13_ADVANCED_WORKFLOWS_GUIDE.md

Advanced Topics:

• GPU acceleration for faster processing
• Custom model tuning for domain-specific accuracy
• Batch processing scripts for multiple files
• Docker deployment for isolated environments

---

🔒 Your data, your device, your control.