Sm Configure

sm-configure: Engine Configuration DSL

Comprehensive reference for configuring SourceMonitor via SourceMonitor.configure.

When to Use

• Adding or modifying settings in config/initializers/source_monitor.rb
• Understanding what configuration options are available
• Debugging configuration-related issues
• Setting up environment-specific overrides

Configuration Entry Point

All configuration lives inside the configure block in the host app's initializer:

SourceMonitor.configure do |config|
  # settings here
end

After the block executes, ModelExtensions.reload! runs automatically to apply any model changes. Restart web and worker processes after changes.

Configuration Sections

The config object (SourceMonitor::Configuration) has 13 sub-sections plus top-level queue/job settings:

See reference/configuration-reference.md for every setting with types, defaults, and examples.

Quick Examples

Queue Configuration

config.queue_namespace = "source_monitor"
config.fetch_queue_name = "source_monitor_fetch"
config.fetch_queue_concurrency = 4

HTTP Client

config.http.timeout = 30
config.http.proxy = ENV["HTTP_PROXY"]
config.http.retry_max = 3

Authentication (Devise)

SourceMonitor is fail-closed by default: with no authenticate_with/authorize_with handler configured, every engine route returns 403 Forbidden. Configure a handler to protect the dashboard (the handler decides access and bypasses the fail-closed guard):

config.authentication.authenticate_with :authenticate_user!
config.authentication.authorize_with ->(c) { c.current_user&.admin? }

For local demos/sandboxes only, opt out of the fail-closed guard (non-production):

config.authentication.open_access = true # default: false -- demo/non-production only

Image Downloads (Active Storage)

config.images.download_to_active_storage = true
config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
config.images.download_timeout = 15
config.images.allowed_content_types = %w[image/jpeg image/png image/webp]

Events

config.events.after_item_created { |e| Notifier.new_item(e.item) }
config.events.register_item_processor ->(ctx) { Indexer.index(ctx.item) }

Model Extensions

config.models.table_name_prefix = "sm_"
config.models.source.include_concern "MyApp::SourceExtension"
config.models.item.validate :custom_check

Favicons (Active Storage)

config.favicons.enabled = true
config.favicons.fetch_timeout = 10
config.favicons.max_download_size = 512 * 1024  # 512 KB
config.favicons.retry_cooldown_days = 14

Realtime

config.realtime.adapter = :redis
config.realtime.redis_url = ENV["REDIS_URL"]

Helper APIs

SourceMonitor.config                         # Current configuration
SourceMonitor.configure { |c| ... }          # Set configuration
SourceMonitor.reset_configuration!           # Revert to defaults (for tests)
SourceMonitor.events                         # Shortcut to config.events
SourceMonitor.queue_name(:fetch)             # Resolved queue name
SourceMonitor.queue_concurrency(:scrape)     # Resolved concurrency
SourceMonitor.mission_control_dashboard_path # Resolved MC path or nil

Key Source Files

References

• reference/configuration-reference.md -- Complete settings reference
• docs/configuration.md -- Official configuration documentation
• lib/generators/source_monitor/install/templates/source_monitor.rb.tt -- Initializer template

Testing

Reset configuration between tests:

setup do
  SourceMonitor.reset_configuration!
end

Test custom configuration:

test "custom queue name" do
  SourceMonitor.configure do |config|
    config.fetch_queue_name = "custom_fetch"
  end
  assert_equal "custom_fetch", SourceMonitor.queue_name(:fetch)
end

Checklist

• [ ] Initializer exists at config/initializers/source_monitor.rb
• [ ] Queue names match config/queue.yml (or config/solid_queue.yml) entries
• [x] Dispatcher config includes recurring_schedule: config/recurring.yml (handled by install generator)
• [ ] Authentication decision made: configure authenticate_with/authorize_with (fail-closed default) OR set config.authentication.open_access = true for demos only
• [ ] HTTP timeouts appropriate for target feeds
• [ ] Retention policy set for production
• [ ] Workers restarted after configuration changes

Configuration Reference

Complete reference for every SourceMonitor configuration setting.

Source: lib/source_monitor/configuration.rb and lib/source_monitor/configuration/*.rb

Top-Level Settings

Defined on SourceMonitor::Configuration:

| maintenance_queue_name | String | "source_monitor_maintenance" | Queue name for maintenance jobs | | maintenance_queue_concurrency | Integer | 1 | Advisory concurrency for maintenance queue |

Methods

---

HTTP Settings (config.http)

Class: SourceMonitor::Configuration::HTTPSettings

config.http.timeout = 30
config.http.proxy = "http://proxy.example.com:8080"
config.http.headers = { "X-Custom" => "value" }
config.http.retry_statuses = [429, 500, 502, 503, 504]

---

Fetching Settings (config.fetching)

Class: SourceMonitor::Configuration::FetchingSettings

Controls adaptive fetch scheduling.

config.fetching.min_interval_minutes = 10
config.fetching.max_interval_minutes = 720  # 12 hours
config.fetching.jitter_percent = 0.15       # +/-15%
config.fetching.scheduler_batch_size = 50   # Increase for larger servers
config.fetching.stale_timeout_minutes = 3   # Faster recovery

---

Health Settings (config.health)

Class: SourceMonitor::Configuration::HealthSettings

Tunes automatic pause/resume heuristics per source.

config.health.window_size = 50
config.health.auto_pause_threshold = 0.1
config.health.auto_pause_cooldown_minutes = 120

---

Scraper Registry (config.scrapers)

Class: SourceMonitor::Configuration::ScraperRegistry

Manages scraper adapter registration. Adapters must inherit from SourceMonitor::Scrapers::Base.

Methods

Names are normalized to lowercase alphanumeric + underscore.

Adapters can be a Class or a String (constantized lazily).

config.scrapers.register(:custom, MyApp::Scrapers::Custom)
config.scrapers.register(:premium, "MyApp::Scrapers::Premium")
config.scrapers.unregister(:readability)

---

Retention Settings (config.retention)

Class: SourceMonitor::Configuration::RetentionSettings

Global defaults inherited by sources with blank retention fields.

config.retention.items_retention_days = 90
config.retention.max_items = 1000
config.retention.strategy = :soft_delete

---

Scraping Settings (config.scraping)

Class: SourceMonitor::Configuration::ScrapingSettings

Values are normalized to positive integers. Set to nil to disable limits.

config.scraping.max_in_flight_per_source = 50
config.scraping.max_bulk_batch_size = 200

---

Events (config.events)

Class: SourceMonitor::Configuration::Events

Register lifecycle callbacks. See the sm-event-handler skill for full event documentation.

Callback Methods

Handlers must respond to #call. Can be a Proc, lambda, or any callable object.

config.events.after_item_created ->(event) { log(event.item) }
config.events.after_fetch_completed do |event|
  StatsTracker.record(event.source, event.status)
end

---

Models (config.models)

Class: SourceMonitor::Configuration::Models

Per-model extension points for host apps.

Model Accessors

Each accessor returns a ModelDefinition with:

config.models.table_name_prefix = "sm_"
config.models.source.include_concern "MyApp::SourceTagging"
config.models.item.validate :check_content_length

---

Realtime Settings (config.realtime)

Class: SourceMonitor::Configuration::RealtimeSettings

Solid Cable Options (config.realtime.solid_cable)

Methods

config.realtime.adapter = :redis
config.realtime.redis_url = "redis://localhost:6379/1"

# Or with Solid Cable tuning:
config.realtime.adapter = :solid_cable
config.realtime.solid_cable.polling_interval = "0.05.seconds"
config.realtime.solid_cable.connects_to = { database: { writing: :cable } }

---

Authentication Settings (config.authentication)

Class: SourceMonitor::Configuration::AuthenticationSettings

Fail-closed by default. When no authenticate_with/authorize_with handler is configured and open_access is false, the engine denies every route with 403 Forbidden (see SourceMonitor::Security::Authentication.access_denied_by_default?). Configuring either handler makes the handler authoritative and bypasses the fail-closed guard.

Methods

Handlers can be:

• Symbol: Invoked as controller.public_send(handler)
• Callable: Called with callable.call(controller) (or instance_exec if arity is 0)

# Symbol handler (Devise)
config.authentication.authenticate_with :authenticate_user!

# Callable handler
config.authentication.authorize_with ->(controller) {
  controller.current_user&.feature_enabled?(:source_monitor)
}

# Block handler (instance_exec on controller)
config.authentication.authorize_with do
  redirect_to root_path unless current_user&.admin?
end

# Open access (demo/non-production only) -- disables the fail-closed guard.
# Leave commented/false in production. A configured handler always wins.
config.authentication.open_access = true # default: false

---

Images Settings (config.images)

Class: SourceMonitor::Configuration::ImagesSettings

Controls background downloading of inline images from feed content to Active Storage.

Prerequisite: The host app must have Active Storage installed (rails active_storage:install + migrations).

Helper Method

# Enable image downloading with custom limits
config.images.download_to_active_storage = true
config.images.max_download_size = 5 * 1024 * 1024  # 5 MB
config.images.download_timeout = 15
config.images.allowed_content_types = %w[image/jpeg image/png image/webp]

When enabled, DownloadContentImagesJob is automatically enqueued after new items are created from feed entries. The job downloads inline <img> images from item.content, attaches them to item_content.images via Active Storage, and rewrites the HTML with Active Storage serving URLs. Failed downloads gracefully preserve the original image URL.

---

Favicons Settings (config.favicons)

Class: SourceMonitor::Configuration::FaviconsSettings

Controls automatic favicon fetching and storage for sources via Active Storage.

Prerequisite: The host app must have Active Storage installed (rails active_storage:install + migrations). Without Active Storage, favicons are silently disabled and colored initials placeholders are shown instead.

Helper Method

# Customize favicon settings
config.favicons.enabled = true
config.favicons.fetch_timeout = 10
config.favicons.max_download_size = 512 * 1024  # 512 KB
config.favicons.retry_cooldown_days = 14
config.favicons.allowed_content_types = %w[image/png image/x-icon image/svg+xml]

When enabled, FaviconFetchJob is automatically enqueued: 1. After a new source is created (via UI or OPML import) with a website_url 2. After a successful feed fetch when the source has no favicon attached and is outside the retry cooldown

The job uses Favicons::Discoverer which tries three strategies in order: 1. Direct /favicon.ico fetch from the source's domain 2. HTML page parsing for <link rel="icon">, <link rel="apple-touch-icon">, and similar tags (prefers largest by sizes attribute) 3. Google Favicon API as a last resort

Failed attempts are tracked in the source's metadata JSONB column (favicon_last_attempted_at) to respect the cooldown period.

---

Environment Variables