.vbw-planning/milestones/07-rails-audit-and-refactoring/07-rails-audit-round-2/.context-dev.md
Not available
Codebase mapping exists in .vbw-planning/codebase/. Key files:
ARCHITECTURE.mdCONCERNS.mdPATTERNS.mdDEPENDENCIES.mdSTRUCTURE.mdCONVENTIONS.mdTESTING.mdSTACK.mdRead CONVENTIONS.md, PATTERNS.md, STRUCTURE.md, and DEPENDENCIES.md first to bootstrap codebase understanding.
.vbw-planning/config.json.vbw-planning/milestones/03-coverage-analysis-quick-wins-critical-path-test-co/STATE.md.vbw-planning/phases/02-feed-reliability/01-SUMMARY.md.vbw-planning/phases/02-feed-reliability/02-SUMMARY.md.vbw-planning/phases/02-feed-reliability/03-SUMMARY.md.vbw-planning/phases/02-feed-reliability/04-SUMMARY.md.vbw-planning/phases/03-dashboard-pagination/03-04-SUMMARY.md.vbw-planning/phases/05-simplify-source-status/01-SUMMARY.md.vbw-planning/phases/05-simplify-source-status/03-SUMMARY.md.vbw-planning/phases/05-simplify-source-status/05-SUMMARY.md.vbw-planning/ROADMAP.mdapp/assets/builds/source_monitor/application.csstest/dummy/config/puma.rbtest/dummy/db/schema.rbtest/dummy/Gemfile.lock.vbw-planning/config.json (51 lines, first 30 shown){
"effort": "thorough",
"autonomy": "standard",
"auto_commit": true,
"planning_tracking": "manual",
"auto_push": "never",
"verification_tier": "standard",
"skill_suggestions": true,
"auto_install_skills": false,
"discovery_questions": true,
"context_compiler": true,
"visual_format": "unicode",
"max_tasks_per_plan": 5,
"prefer_teams": "auto",
"branch_per_milestone": false,
"plain_summary": true,
"active_profile": "default",
"custom_profiles": {},
"model_profile": "quality",
"model_overrides": {},
"agent_max_turns": {
"scout": 15,
"qa": 25,
"architect": 30,
"debugger": 80,
"lead": 50,
"dev": 75
},
"qa_skip_agents": [
"docs"
.vbw-planning/milestones/03-coverage-analysis-quick-wins-critical-path-test-co/STATE.md (64 lines, first 30 shown)<!-- VBW STATE TEMPLATE (ARTF-05) -- Session dashboard, auto-updated -->
<!-- Updated after each plan completion and at checkpoints -->
# Project State
## Project Reference
See: .vbw-planning/PROJECT.md (updated 2026-02-09)
**Core value:** Drop-in Rails engine for feed monitoring, content scraping, and operational dashboards.
**Current focus:** All phases complete
## Current Position
Phase: 4 of 4 (Code Quality & Conventions Cleanup)
Plan: 3 of 3 in current phase
Status: Built
Last activity: 2026-02-10 -- Phase 4 complete (3/3 plans done)
Progress: [##########] 100%
## Codebase Profile
- **Total source files:** 535
- **Primary language:** Ruby (330 files)
- **Templates:** ERB (48 files)
- **Tests:** 137 test files detected
- **Test suite:** 841 runs, 2776 assertions, 0 failures (up from 473 runs in Phase 1)
- **Coverage:** 86.97% line, 58.84% branch (510 uncovered lines, down from 2117)
- **CI/CD:** GitHub Actions (1 workflow)
.vbw-planning/ROADMAP.md (135 lines, first 30 shown)# Roadmap
**Milestone:** rails-audit-and-modal
## Phases
- [x] Phase 01: Quick Wins & Security
- [x] Phase 02: Model Layer Hardening
- [x] Phase 03: Controller & Route Refactoring
- [x] Phase 04: Job & Pipeline Reliability
- [x] Phase 05: View Layer Extraction
- [x] Phase 06: Test Infrastructure
- [ ] Phase 07: Rails Audit Round 2
## Phase Details
### Phase 01: Quick Wins & Security
**Goal:** Fix high-priority security issues and low-effort quick wins from the Rails audit (findings M1, M3, M5, M12, C6, C9, V2, V8, V10, V11).
**Success Criteria:**
- ImportHistoryDismissalsController scoped to current user
- DashboardController uses explicit parameter allowlist (no .permit!)
- ScrapeLog has by_source, by_status, by_item scopes
- Scrape status badge logic extracted to helper
- MutationObserver disconnect leak fixed
**Plans:**
- [x] Plan 01: Security & Controller Fixes
- [x] Plan 02: Model & Job Fixes
app/assets/builds/source_monitor/application.css (2265 lines, first 30 shown)*, ::before, ::after {
--tw-border-spacing-x: 0;
--tw-border-spacing-y: 0;
--tw-translate-x: 0;
--tw-translate-y: 0;
--tw-rotate: 0;
--tw-skew-x: 0;
--tw-skew-y: 0;
--tw-scale-x: 1;
--tw-scale-y: 1;
--tw-pan-x: ;
--tw-pan-y: ;
--tw-pinch-zoom: ;
--tw-scroll-snap-strictness: proximity;
--tw-gradient-from-position: ;
--tw-gradient-via-position: ;
--tw-gradient-to-position: ;
--tw-ordinal: ;
--tw-slashed-zero: ;
--tw-numeric-figure: ;
--tw-numeric-spacing: ;
--tw-numeric-fraction: ;
--tw-ring-inset: ;
--tw-ring-offset-width: 0px;
--tw-ring-offset-color: #fff;
--tw-ring-color: rgb(59 130 246 / 0.5);
--tw-ring-offset-shadow: 0 0 #0000;
--tw-ring-shadow: 0 0 #0000;
--tw-shadow: 0 0 #0000;
--tw-shadow-colored: 0 0 #0000;
test/dummy/config/puma.rb (44 lines)# frozen_string_literal: true
# This configuration file will be evaluated by Puma. The top-level methods that
# are invoked here are part of Puma's configuration DSL. For more information
# about methods provided by the DSL, see https://puma.io/puma/Puma/DSL.html.
#
# Puma starts a configurable number of processes (workers) and each process
# serves each request in a thread from an internal thread pool.
#
# You can control the number of workers using ENV["WEB_CONCURRENCY"]. You
# should only set this value when you want to run 2 or more workers. The
# default is already 1.
#
# The ideal number of threads per worker depends both on how much time the
# application spends waiting for IO operations and on how much you wish to
# prioritize throughput over latency.
#
# As a rule of thumb, increasing the number of threads will increase how much
# traffic a given process can handle (throughput), but due to CRuby's
# Global VM Lock (GVL) it has diminishing returns and will degrade the
# response time (latency) of the application.
#
# The default is set to 3 threads as it's deemed a decent compromise between
# throughput and latency for the average Rails application.
#
# Any libraries that use a connection pool or another resource pool should
# be configured to provide at least as many connections as the number of
# threads. This includes Active Record's `pool` parameter in `database.yml`.
threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)
threads threads_count, threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
port ENV.fetch("PORT", 3000)
# Allow puma to be restarted by `bin/rails restart` command.
plugin :tmp_restart
# Specify the PID file. Defaults to tmp/pids/server.pid in development.
# In other environments, only set the PID file if requested.
pidfile ENV["PIDFILE"] if ENV["PIDFILE"]
# Enable YJIT if available (Ruby 3.3+ / 4.0+)
# Must be done at boot, not in before_fork (which only runs in cluster mode).
RubyVM::YJIT.enable if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
test/dummy/db/schema.rb (449 lines, first 30 shown)# This file is auto-generated from the current state of the database. Instead
# of editing this file, please use the migrations feature of Active Record to
# incrementally modify your database, and then regenerate this schema definition.
#
# This file is the source Rails uses to define your schema when running `bin/rails
# db:schema:load`. When creating a new database, `bin/rails db:schema:load` tends to
# be faster and is potentially less error prone than running all of your
# migrations from scratch. Old migrations may fail to apply correctly if those
# migrations use external dependencies or application code.
#
# It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[8.1].define(version: 2026_03_13_120000) do
# These are extensions that must be enabled in order to support this database
enable_extension "pg_catalog.plpgsql"
create_table "active_storage_attachments", force: :cascade do |t|
t.bigint "blob_id", null: false
t.datetime "created_at", null: false
t.string "name", null: false
t.bigint "record_id", null: false
t.string "record_type", null: false
t.index ["blob_id"], name: "index_active_storage_attachments_on_blob_id"
t.index ["record_type", "record_id", "name", "blob_id"], name: "index_active_storage_attachments_uniqueness", unique: true
end
create_table "active_storage_blobs", force: :cascade do |t|
t.bigint "byte_size", null: false
t.string "checksum"
t.string "content_type"
test/dummy/Gemfile.lock (415 lines, first 30 shown)PATH
remote: ../..
specs:
source_monitor (0.11.1)
cssbundling-rails (~> 1.4)
faraday (~> 2.9)
faraday-follow_redirects (~> 0.4)
faraday-gzip (~> 3.0)
faraday-retry (~> 2.2)
feedjira (>= 3.2, < 5.0)
jsbundling-rails (~> 1.3)
nokolexbor (~> 0.5)
rails (>= 8.0.3, < 10.0)
ransack (~> 4.2)
ruby-readability (~> 0.7)
solid_cable (>= 3.0, < 4.0)
solid_queue (>= 0.3, < 3.0)
turbo-rails (~> 2.0)
view_component (>= 3.0, < 4.0)
GEM
remote: https://rubygems.org/
specs:
action_text-trix (2.1.16)
railties
actioncable (8.1.2)
actionpack (= 8.1.2)
activesupport (= 8.1.2)
nio4r (~> 2.0)
websocket-driver (>= 0.6.1)
phase: 7 plan: 01 title: Model Correctness & Data Integrity type: execute wave: 1 depends_on: [] cross_phase_deps: [] autonomous: true effort_override: thorough skills_used: [sm-domain-model, sm-engine-test, tdd-cycle] files_modified:
Key context: LogCleanupJob uses batch.delete_all on FetchLog/ScrapeLog which skips dependent: :destroy callbacks, orphaning LogEntry records. Source has health_status default "working" in model but "healthy" in DB schema. Three log models duplicate identical sync_log_entry callbacks that belong in their shared Loggable concern. Item#soft_delete! decrements counter cache but has no restore! to re-increment. </context> <tasks> <task type="auto"> <name>Fix LogCleanupJob orphaned LogEntry records (H1)</name> <files> app/jobs/source_monitor/log_cleanup_job.rb test/jobs/source_monitor/log_cleanup_job_test.rb </files> <action> In LogCleanupJob, before each batch.delete_all call on FetchLog/ScrapeLog/HealthCheckLog, first delete the corresponding LogEntry records:
# Before deleting FetchLog batch:
SourceMonitor::LogEntry.where(loggable_type: "SourceMonitor::FetchLog", loggable_id: batch.select(:id)).delete_all
batch.delete_all
Apply this pattern to all three log type cleanup sections. Add tests that:
Note: Check what values the health module actually uses. The model says default "working", health module may use "healthy", "declining", "improving", "failing". Include all values that appear in the codebase. </action> <verify> PARALLEL_WORKERS=1 bin/rails test test/models/source_monitor/source_test.rb </verify> <done> Source has HEALTH_STATUS_VALUES constant, inclusion validation on health_status, migration aligns DB default to "working", scraping_enabled/scraping_disabled scopes added. </done> </task> <task type="auto"> <name>Add Item#restore! and fix counter cache symmetry (M3)</name> <files> app/models/source_monitor/item.rb test/models/source_monitor/item_test.rb </files> <action>
restore! method to Item that sets deleted_at to nil via update_columns and calls Source.increment_counter(:items_count, source_id). This is the symmetric counterpart to soft_delete!.5 parallel agents audited the entire SourceMonitor engine codebase across models, controllers, services/jobs/pipeline, views/frontend, and testing layers.
| Severity | Remaining | Already Fixed |
|---|---|---|
| HIGH | 6 | 0 |
| MEDIUM | 17 | 10 |
| LOW | 21 | 9 |
| Total | 44 | 19 |
Overall verdict: The codebase is well-structured for an engine of this complexity. Routes follow CRUD conventions, concerns are well-scoped, and the Hotwire frontend has solid fundamentals. Recent milestone work (phases 01-06) already resolved ~19 findings. The remaining gaps are primarily business logic in jobs (violates "shallow jobs" convention) and duplicated logic across pipeline layers.
Note: 19 findings from the initial 63 were already addressed by commits in the recent ui-fixes-and-smart-scraping milestone. These are marked
strikethroughbelow. The counts above reflect only unresolved findings.
app/jobs/source_monitor/log_cleanup_job.rb:42-49batch.delete_all on FetchLog/ScrapeLog records. These models have has_one :log_entry, dependent: :destroy, but delete_all skips callbacks, orphaning LogEntry records.loggable_type/loggable_id, then delete the log records. Or use destroy_in_batches.dependent: :destroy declaration shows cascade was intended.app/jobs/source_monitor/import_opml_job.rb:14-157SourceMonitor::ImportSessions::OPMLImporter service. The job becomes a 5-line delegation.app/jobs/source_monitor/scrape_item_job.rb:14-57mark_processing!, mark_failed!, clear_inflight!), and re-enqueues itself with a delay.Scraping::Runner. Job becomes a one-liner delegation.time_until_scrape_allowed duplicates near-identical logic in Scraping::Enqueuer#time_rate_limited?. Two places to maintain the same business rule.app/jobs/source_monitor/download_content_images_job.rb:17-49SourceMonitor::Images::Processor. Job delegates with a single call.app/jobs/source_monitor/scrape_item_job.rb:47-57 and lib/source_monitor/scraping/enqueuer.rb:129-143min_scrape_interval. The Enqueuer defers the job, and the Job re-checks and defers again.ScrapeItemJob. The Enqueuer already handles deferral at enqueue time. If race conditions are a concern, have the job delegate to a runner that calls the Enqueuer.Source.destroy_all in pagination tests is not parallel-safetest/controllers/source_monitor/sources_controller_test.rb:252,264,276,286,297,311,322Source.destroy_all to get a clean slate for pagination counting. With thread-based parallelism, this can race with other threads.TEST_CONVENTIONS.md section 6.health_status default mismatch between model and databaseapp/models/source_monitor/source.rb:37 vs db/schema.rb:384attribute :health_status, :string, default: "working" but schema has default: "healthy". A Source created in Ruby gets "working", one via raw SQL gets "healthy".validates :health_status, inclusion: { in: HEALTH_STATUS_VALUES }.health_status validationapp/models/source_monitor/source.rb:44-51fetch_status has an inclusion validation; health_status has none. Any arbitrary string can be stored.HEALTH_STATUS_VALUES = %w[healthy working declining failing].freeze and validates :health_status, inclusion: { in: HEALTH_STATUS_VALUES }.Item#soft_delete! counter cache fragilityapp/models/source_monitor/item.rb:69-83Source.decrement_counter(:items_count, source_id) after update_columns. No corresponding restore! method to re-increment.restore! method for symmetry. Consider extracting soft-delete into a concern.sync_log_entry callback across 3 log modelsapp/models/source_monitor/fetch_log.rb:28, scrape_log.rb:20, health_check_log.rb:20after_save :sync_log_entry callbacks.Loggable concern.rescue_from ActiveRecord::RecordNotFoundapp/controllers/source_monitor/application_controller.rbset_source across 7 controllerssource_fetches_controller.rb, source_retries_controller.rb, source_bulk_scrapes_controller.rb, source_health_checks_controller.rb, source_health_resets_controller.rb, source_favicon_fetches_controller.rb, source_scrape_tests_controller.rbdef set_source; @source = Source.find(params[:source_id]); end.SetSource concern.fallback_user_id creates users in host-app tablesapp/controllers/source_monitor/import_sessions_controller.rb:244-276Rails.env.development? or remove entirely. An engine should never create records in host-app tables.app/controllers/source_monitor/import_sessions/opml_parser.rb (128 lines), entry_annotation.rb (187 lines), health_check_management.rb (112 lines), bulk_configuration.rb (106 lines)lib/ or app/services/ classes. Controller concerns become thin wrappers.SourcesController#index has 47 lines of query orchestrationa6d7148 (extract sources index metrics) + 795b7b8 (SourcesFilterPresenter) + cafefc2 (FilterDropdownComponent)BulkScrapeEnablementsController contains business logicapp/controllers/source_monitor/bulk_scrape_enablements_controller.rb:13-19update_all with field combination for enabling scraping is in the controller.Source.enable_scraping!(ids) class method.update_column usage in ImportSessions flowapp/controllers/source_monitor/import_sessions_controller.rb (11 calls)current_step and selected_source_ids changes.ImportSession#advance_to!(step).app/jobs/source_monitor/favicon_fetch_job.rb:17-42SourceUpdater#enqueue_favicon_fetch_if_needed.Favicons::FetchService. Consolidate cooldown in Favicons::CooldownCheck.app/jobs/source_monitor/import_session_health_check_job.rb:18-63ImportSessions::HealthCheckUpdater.app/jobs/source_monitor/source_health_check_job.rb:29-83toast_payload, broadcast_outcome, trigger_fetch_if_degraded are all presentation/side-effect logic.Health::SourceHealthCheckOrchestrator.e03723d added Result structs to completion handlers; 5bd538a wired Result usage in FetchRunnerFeedFetcher::Result still lacks success?. No shared base SourceMonitor::Result class exists yet.4ff8884 (extract FetchFeedJob retry orchestrator service)feed_fetcher.rb:331, scraping/state.rb:68, source_health_check_job.rb:46-47rescue StandardError => nil silently swallows failures.Rails.logger.warn in rescue blocks.lib/source_monitor/fetching/stalled_fetch_reconciler.rb:107where("arguments::jsonb -> 'arguments' ->> 0 = ?", source.id.to_s) — fragile if SolidQueue changes serialization.items/_details.html.erb:6-7, sources/_bulk_scrape_modal.html.erb:3-11item.scrape_logs.order(...).limit(5) and similar queries directly in ERB.sources/_details.html.erb was partially addressed by SourceDetailsPresenter (7c2604b), but items and bulk scrape modal still have inline queries.sources/_row.html.erb, sources/_details.html.erb, dashboard/_recent_activity.html.erb, items/_details.html.erb, items/index.html.erb, logs/index.html.erb<span class="inline-flex items-center rounded-full ..."> with conditional spinners.StatusBadgeComponent.IconComponent was added (caa4e69) for SVG icons, but status badges are a separate pattern that still needs extraction.SourceDetailsPresenter added in 7c2604bItemDetailsPresenter, FetchLogPresenter, ScrapeLogPresenter, SourceRowPresenter still missing.role="dialog" and aria-modalsources/_bulk_scrape_modal.html.erb, sources/_bulk_scrape_enable_modal.html.erb<div> elements.role="dialog", aria-modal="true", aria-labelledby pointing to heading.app/assets/javascripts/source_monitor/controllers/modal_controller.jsinert attribute on background elements.app/views/source_monitor/logs/index.html.erbform_with uses local: true disabling Turbo. Full page reload on filter.font-semibold vs font-medium, padding variations.@apply CSS classes or a ButtonComponent.ApplicationHelper is 333 lines with mixed concernsapp/helpers/source_monitor/application_helper.rbStatusBadgeHelper, FaviconHelper, PaginationHelper, FetchIntervalHelper, ExternalLinkHelper.SourcesFilterPresenter (795b7b8) and FilterDropdownComponent (cafefc2) extracted some filter logic, but the helper itself is still large.create_item! factory underused18692f6 centralized factory helpers into ModelFactories moduleItem.create! instead of the now-centralized create_item!. Migration to the shared factories is incomplete.new action delegates to create in ImportSessionsController (GET creates records)import_sessions_controller.rb:35-37 — quickBulkScrapeEnablementsController accesses params without strong params wrapperbulk_scrape_enablements_controller.rb:6 — quickSourceScrapeTestsController#create builds result hash inline (should be presenter)source_scrape_tests_controller.rb:14-24 — shortSourceHealthChecksController embeds Tailwind classes in controllersource_health_checks_controller.rb:25-31 — quickrescue StandardError in action controllers6bcd0ac and 19bb3b8 (transient vs fatal error classification in FaviconFetchJob and DownloadContentImagesJob) + 911c17e (deadlock rescue + error logging)SanitizesSearchParams uses to_unsafe_h without documentationconcerns/source_monitor/sanitizes_search_params.rb:45 — quickSourcesController#update contains conditional job-enqueue logicsources_controller.rb:94-109 — shortSource::SearchableAttributes concernsource.rb:79-103 — shortscraping_enabled / scraping_disabled scopessource.rb — quickItemContent#compute_feed_word_count reaches through association (minor Demeter violation)item_content.rb:33-39 — quickImportHistory missing chronological validation and JSONB attribute declarationsimport_history.rb — quickFetchFeedJob#should_run? scheduling logic4ff8884 extracted retry orchestrator, reducing job logic. should_run? guard still exists but is simpler.feed_fetcher.rb:393-404, item_creator.rb:179-197 — mediumfeed_fetcher.rb:30-36 — quickImages::Downloader creates raw Faraday connection instead of HTTP.clientimages/downloader.rb:46-58 — quickLogs::Query is good; Scheduler queries could be extractedscheduler.rb:55-89, scraping/scheduler.rb:38-45 — shortfetching/cloudflare_bypass.rb:39-49 — quick_pagination.html.erb partialitems/index.html.erb:124-146, logs/index.html.erb:186-209 — shortsource_scrape_tests/show.html.erb:8-56, _result.html.erb:12-60 — quickPanelComponent)_form_errors partial)sources/new.html.erb:4-13, edit.html.erb:4-13 — quick491fae1 (simplify dropdown controller and remove JS globals)applyLevelDelay() method15e7d53 (remove dead JS error delay override and document constants)aria-label, should use IconComponent4c56789 replaced inline SVGs with IconComponent. Check if this specific dismiss button was included.FilterDropdownComponent uses inline onchange instead of Stimulus actionfilter_dropdown_component.rb:48,55 — shortconfigure_authentication helper across 4 test files54617b8 created SystemTestHelpers module with shared purge method. Some lib tests may still inline it.| # | Finding | Severity | Effort | Category |
|---|---|---|---|---|
| 1 | H1 — Fix LogCleanupJob orphaned LogEntry records | HIGH | short | Data integrity |
| 2 | H5 — Remove duplicated scrape rate-limiting from ScrapeItemJob | HIGH | quick | DRY |
| 3 | M6 — Extract set_source into shared concern | MEDIUM | quick | DRY |
| 4 | M1+M2 — Align health_status default + add validation | MEDIUM | quick | Correctness |
| 5 | M4 — Move sync_log_entry callback into Loggable concern | MEDIUM | quick | DRY |
| 6 | M5 — Add rescue_from RecordNotFound | MEDIUM | short | Robustness |
| 7 | H4 — Extract DownloadContentImagesJob orchestration | HIGH | short | Convention |
| 8 | H6 — Fix pagination test parallel-safety | HIGH | medium | Test reliability |
| 9 | H2 — Extract ImportOpmlJob business logic to service | HIGH | medium | Convention |
| 10 | M20 — Create StatusBadgeComponent | MEDIUM | medium | DRY/Consistency |
Previously in Top 10, now resolved: M9 (SourcesController#index metrics — extracted to presenters), M16 (retry logic consolidation — extracted to RetryOrchestrator service)
The audit identified many areas where the codebase excels:
source_fetches, source_retries, etc.)inverse_of, dependent: :destroy, and thorough indexingModelFactories) have good defaults with SecureRandom for parallel safetyTEST_CONVENTIONS.md with clear guidanceSourceMonitor.reset_configuration!if defined?(ActiveStorage) checksSources::Params.sanitize with explicit allowlistGenerated by /rails-audit command. Re-run to refresh findings.