Configuration Hot Reload¶
Navigator supports live configuration reloading without restarting the server or interrupting active requests. This enables zero-downtime configuration updates in production.
How It Works¶
When Navigator receives a SIGHUP
signal, it:
- Parses new configuration from the YAML file
- Updates routing rules and application settings
- Starts new managed processes if added to config
- Keeps existing Rails processes running
- Applies changes to new requests immediately
Triggering Reload¶
CLI Command (Recommended)¶
Unix Signal¶
# Send SIGHUP signal directly
kill -HUP $(cat /tmp/navigator.pid)
# Or using process name
pkill -HUP navigator
systemd Integration¶
What Gets Reloaded¶
✅ Reloaded Without Restart¶
- Application routing rules: Path mappings, tenant configurations
- Authentication settings: htpasswd files, public paths
- Static file configurations: Directories, caching, try files
- Managed process definitions: New processes started
- Environment variables: For new processes only
- Fly-Replay rules: Routing patterns and targets
- Reverse proxy settings: Target URLs and headers
❌ Requires Full Restart¶
- Server listen port: Cannot change port without restart
- Process pool limits:
max_size
changes require restart - Working directories: Of existing Rails processes
- PID file location: Set at startup only
Configuration Example¶
Before Reload¶
server:
listen: 3000
applications:
tenants:
- name: app1
path: /app1/
working_dir: /var/www/app1
managed_processes:
- name: redis
command: redis-server
After Configuration Update¶
server:
listen: 3000 # Same - no restart needed
applications:
tenants:
- name: app1
path: /app1/
working_dir: /var/www/app1
# New tenant - will be available immediately
- name: app2
path: /app2/
working_dir: /var/www/app2
managed_processes:
- name: redis
command: redis-server
# New process - will be started
- name: sidekiq
command: bundle
args: [exec, sidekiq]
working_dir: /var/www/app1
Reload Process¶
# 1. Update configuration file
vim /etc/navigator/config.yml
# 2. Reload Navigator
navigator -s reload
# 3. Verify new tenant is available
curl http://localhost:3000/app2/
# 4. Check new process is running
ps aux | grep sidekiq
Use Cases¶
1. Adding New Tenants¶
Add new applications without downtime:
# Add tenant to config
echo " - name: new-client
path: /clients/new-client/
working_dir: /var/www/new-client" >> config.yml
# Reload
navigator -s reload
# New tenant immediately available
curl http://localhost:3000/clients/new-client/
2. Authentication Updates¶
Update htpasswd files and reload:
# Add new user
htpasswd /etc/navigator/htpasswd newuser
# Update public paths in config
# ...
# Reload authentication settings
navigator -s reload
3. Static File Changes¶
Add new static file directories:
# Add to config
static:
directories:
- path: /assets/
root: public/assets/
- path: /uploads/ # New directory
root: storage/uploads/
cache: 3600
4. Managed Process Updates¶
Add background services without restart:
managed_processes:
- name: redis
command: redis-server
- name: worker-queue # New worker
command: ./worker.sh
auto_restart: true
Production Workflow¶
Safe Deployment Pattern¶
#!/bin/bash
# production-reload.sh
# 1. Backup current config
cp /etc/navigator/config.yml /etc/navigator/config.yml.backup
# 2. Validate new configuration
if navigator --validate /tmp/new-config.yml; then
echo "✓ Configuration valid"
else
echo "✗ Configuration invalid, aborting"
exit 1
fi
# 3. Update configuration
cp /tmp/new-config.yml /etc/navigator/config.yml
# 4. Reload Navigator
navigator -s reload
# 5. Verify reload succeeded
if ps aux | grep -q navigator; then
echo "✓ Reload successful"
else
echo "✗ Reload failed, restoring backup"
cp /etc/navigator/config.yml.backup /etc/navigator/config.yml
systemctl restart navigator
exit 1
fi
# 6. Test new configuration
curl -f http://localhost:3000/health || {
echo "✗ Health check failed"
exit 1
}
echo "✓ Production reload completed successfully"
CI/CD Integration¶
# .github/workflows/deploy.yml
name: Deploy Configuration
on:
push:
paths:
- 'config/production.yml'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Validate configuration
run: navigator --validate config/production.yml
- name: Deploy to server
run: |
scp config/production.yml server:/tmp/new-config.yml
ssh server 'sudo ./production-reload.sh'
Monitoring Reloads¶
Logging¶
Navigator logs reload events:
Example log output:
INFO Received SIGHUP, reloading configuration
INFO Configuration reloaded successfully
INFO Starting new managed process name=sidekiq
INFO New tenant added name=app2 path=/app2/
Health Checks¶
Verify reload success:
#!/bin/bash
# check-reload.sh
# Send reload signal
navigator -s reload
# Wait for reload to complete
sleep 2
# Check if Navigator is still running
if ! ps aux | grep -q navigator; then
echo "ERROR: Navigator stopped after reload"
exit 1
fi
# Test application response
if curl -f http://localhost:3000/health; then
echo "SUCCESS: Reload completed"
else
echo "ERROR: Application not responding after reload"
exit 1
fi
Metrics Collection¶
Track reload frequency and success:
# Count reloads in logs
grep "reloading configuration" /var/log/navigator.log | wc -l
# Check for reload errors
grep -E "(reload.*error|failed.*reload)" /var/log/navigator.log
Troubleshooting¶
Reload Doesn't Take Effect¶
Problem: Configuration changes not applied after reload
Causes: - Invalid YAML syntax - File permissions issues - Navigator didn't receive signal
Solutions:
# 1. Validate configuration
navigator --validate config.yml
# 2. Check file permissions
ls -la config.yml
# 3. Verify Navigator is running
ps aux | grep navigator
# 4. Check PID file exists
cat /tmp/navigator.pid
# 5. Send signal manually
kill -HUP $(cat /tmp/navigator.pid)
Process Fails to Start After Reload¶
Problem: New managed process doesn't start
Causes: - Command not found - Working directory doesn't exist - Environment variables missing
Solutions:
# Test command manually
cd /working/directory
./command-name
# Check Navigator logs
tail -f /var/log/navigator.log
# Verify working directory exists
ls -la /working/directory
Existing Requests Affected¶
Problem: Active requests fail after reload
This should NOT happen - reload is designed to be zero-downtime. If this occurs:
- Check configuration: Ensure no breaking changes
- Review logs: Look for error messages
- Consider restart: May need full restart for certain changes
Best Practices¶
1. Always Validate First¶
2. Use Configuration Management¶
# Version control your configs
git add config/production.yml
git commit -m "Add new tenant"
git push
# Deploy with automation
ansible-playbook deploy-config.yml
3. Monitor Reload Success¶
4. Gradual Rollout¶
# Add one tenant at a time
tenants:
- name: existing-app
path: /
- name: new-app-pilot # Test with limited users first
path: /pilot/
Limitations¶
- Port changes: Require full restart
- Process pool size: Changes need restart
- Existing processes: Keep running with old config
- SSL certificates: May need restart depending on implementation