Back to Source Monitor

Phase 5 Verification: Active Storage Image Downloads

.vbw-planning/milestones/generator-enhancements/phases/05-active-storage-images/05-VERIFICATION.md

0.13.010.2 KB
Original Source

Phase 5 Verification: Active Storage Image Downloads

Must-Have Checks

PLAN-01: images-config-model-rewriter

#Truth/ConditionStatusEvidence
1PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/configuration/images_settings_test.rb exits 0 with 0 failuresPASS15 runs, 21 assertions, 0 failures
2PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/images/content_rewriter_test.rb exits 0 with 0 failuresPASS27 runs, 41 assertions, 0 failures
3PARALLEL_WORKERS=1 bin/rails test test/models/source_monitor/item_content_test.rb exits 0 with 0 failuresPASS6 runs, 10 assertions, 0 failures
4bin/rubocop on PLAN-01 files exits 0 with no offensesPASS3 files inspected, 0 offenses
5SourceMonitor.config.images returns an ImagesSettings instancePASSattr_reader :images in configuration.rb, @images = ImagesSettings.new
6SourceMonitor.config.images.download_to_active_storage defaults to falsePASSDEFAULT in ImagesSettings.reset! sets to false
7SourceMonitor.reset_configuration! resets images settings to defaultsPASSTested in images_settings_test.rb (15 tests pass)
8ContentRewriter.new(html).image_urls returns an array of absolute image URLsPASSContentRewriter#image_urls method verified (27 tests pass)
9ContentRewriter.new(html).rewrite { |url| new_url } replaces img src attributesPASSContentRewriter#rewrite method verified (27 tests pass)
10ItemContent responds to images (has_many_attached)PASShas_many_attached :images in item_content.rb (6 tests pass)

PLAN-02: download-job-integration-docs

#Truth/ConditionStatusEvidence
11PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/images/downloader_test.rb exits 0 with 0 failuresPASS11 runs, 18 assertions, 0 failures
12PARALLEL_WORKERS=1 bin/rails test test/jobs/source_monitor/download_content_images_job_test.rb exits 0 with 0 failuresPASS10 runs, 29 assertions, 0 failures
13PARALLEL_WORKERS=1 bin/rails test test/lib/source_monitor/fetching/feed_fetcher/entry_processor_test.rb exits 0 with 0 failuresPASS5 runs, 8 assertions, 0 failures
14bin/rubocop on PLAN-02 files exits 0 with no offensesPASS3 files inspected, 0 offenses
15When config.images.download_to_active_storage is false (default), no job enqueuedPASSentry_processor_test.rb verifies default behavior
16When config.images.download_to_active_storage is true, job enqueued for new items with HTMLPASSentry_processor_test.rb verifies enabled behavior
17DownloadContentImagesJob downloads images, attaches to ItemContent, rewrites item.contentPASSdownload_content_images_job_test.rb (10 tests verify full pipeline)
18Images that fail to download preserve original URLs (graceful fallback)PASSdownload_content_images_job_test.rb tests individual failure handling
19Images larger than max_download_size are skippedPASSdownloader_test.rb tests size validation
20Images with disallowed content types are skippedPASSdownloader_test.rb tests content type validation
21sm-configure skill documents the new config.images sectionPASSconfig.images in SKILL.md and configuration-reference.md

Artifact Checks

ArtifactExistsContainsStatus
lib/source_monitor/configuration/images_settings.rbYESclass ImagesSettingsPASS
lib/source_monitor/images/content_rewriter.rbYESclass ContentRewriterPASS
test/lib/source_monitor/configuration/images_settings_test.rbYESclass ImagesSettingsTest (15 tests)PASS
test/lib/source_monitor/images/content_rewriter_test.rbYESclass ContentRewriterTest (27 tests)PASS
test/models/source_monitor/item_content_test.rbYES6 tests for has_many_attachedPASS
lib/source_monitor/images/downloader.rbYESclass DownloaderPASS
app/jobs/source_monitor/download_content_images_job.rbYESclass DownloadContentImagesJobPASS
test/lib/source_monitor/images/downloader_test.rbYES11 tests for DownloaderPASS
test/jobs/source_monitor/download_content_images_job_test.rbYES10 tests for jobPASS
test/lib/source_monitor/fetching/feed_fetcher/entry_processor_test.rbYES5 tests for integration hookPASS
.claude/skills/sm-configure/SKILL.mdYESconfig.images sectionPASS
.claude/skills/sm-configure/reference/configuration-reference.mdYESImagesSettings documentationPASS
FromToViaStatus
images_settings.rb#download_to_active_storageREQ-24Configurable option defaults to falsePASS
content_rewriter.rb#image_urlsREQ-24Detects inline images in item contentPASS
content_rewriter.rb#rewriteREQ-24Replaces original URLs with Active Storage URLsPASS
item_content.rb#has_many_attachedREQ-24Images attached to ItemContent via Active StoragePASS
download_content_images_job.rb#performREQ-24Downloads inline images to Active StoragePASS
entry_processor.rb#enqueue_image_downloadREQ-24Enqueues download job when config enabledPASS
downloader.rbREQ-24Validates size and content type before downloadPASS
sm-configure/SKILL.mdREQ-24Configuration documented in skillPASS

Convention Compliance

ConventionFileStatusDetail
frozen_string_literalimages_settings.rbPASSPresent in line 1
frozen_string_literalcontent_rewriter.rbPASSPresent in line 1
frozen_string_literaldownloader.rbPASSPresent in line 1
frozen_string_literaldownload_content_images_job.rbPASSPresent in line 1
Test coverageImagesSettingsPASS15 tests covering defaults, accessors, reset, download_enabled?
Test coverageContentRewriterPASS27 tests covering extraction, rewriting, edge cases
Test coverageDownloaderPASS11 tests covering success/failure modes
Test coverageDownloadContentImagesJobPASS10 tests covering full pipeline and failures
Test coverageEntryProcessor integrationPASS5 tests covering enabled/disabled/failure modes
NamingImagesSettingsPASSFollows Configuration sub-class pattern
NamingImages modulePASSFollows engine module structure
AutoloadingImages::ContentRewriterPASSAutoloaded in lib/source_monitor.rb
AutoloadingImages::DownloaderPASSAutoloaded in lib/source_monitor.rb

Anti-Pattern Scan

PatternFoundLocationSeverity
TODO/HACK/FIXME/XXXNOlib/source_monitor/images/*N/A
TODO/HACK/FIXME/XXXNOapp/jobs/source_monitor/download_content_images_job.rbN/A
.html_safeNOlib/source_monitor/images/*N/A
Unsafe HTML renderingNOAll new filesN/A

Requirement Mapping

RequirementPlan RefArtifact EvidenceStatus
REQ-24: Configurable image download optionPLAN-01ImagesSettings with download_to_active_storage (default false)PASS
REQ-24: Image detection in contentPLAN-01ContentRewriter#image_urls extracts URLs from HTMLPASS
REQ-24: Active Storage attachmentPLAN-01ItemContent has_many_attached :imagesPASS
REQ-24: URL rewriting in contentPLAN-01ContentRewriter#rewrite replaces src attributesPASS
REQ-24: Image download servicePLAN-02Downloader downloads, validates size/content-typePASS
REQ-24: Background job orchestrationPLAN-02DownloadContentImagesJob performs full pipelinePASS
REQ-24: Pipeline integrationPLAN-02EntryProcessor enqueues job for new itemsPASS
REQ-24: Graceful fallback on failuresPLAN-02Failed downloads preserve original URLsPASS
REQ-24: Zero behavior change when disabledPLAN-02Default config (false) = no jobs enqueuedPASS
REQ-24: DocumentationPLAN-02sm-configure skill updated with config.imagesPASS

Summary

Tier: standard

Result: PASS

Passed: 34/34

Failed: None

Highlights

  • Configuration layer: ImagesSettings with 4 configurable attributes (download_to_active_storage defaults to false, max_download_size 10MB, download_timeout 30s, allowed_content_types 5 image MIME types). Fully integrated into SourceMonitor.config.images.

  • Model layer: ItemContent has_many_attached :images via Active Storage. Active Storage tables installed in dummy app for testing.

  • HTML processing: ContentRewriter uses Nokolexbor to extract image URLs and rewrite img src attributes. Handles relative URLs, data: URIs, malformed URLs gracefully.

  • Download service: Downloader validates content type and size, derives filenames, returns nil on any failure for graceful fallback.

  • Background job: DownloadContentImagesJob orchestrates the full pipeline: extract URLs, download images, attach to ItemContent, rewrite item.content with Active Storage blob paths. Idempotent (skips if images already attached). Runs on fetch queue.

  • Integration hook: EntryProcessor calls enqueue_image_download after item creation. Only fires when config enabled and item has HTML content. Wrapped in rescue to prevent feed processing breakage.

  • Documentation: sm-configure skill and reference updated with full config.images section.

  • Test coverage: 74 new tests (15 + 27 + 6 + 11 + 10 + 5) covering all scenarios. Full suite: 967 runs, 3100 assertions, 0 failures.

  • Code quality: RuboCop 389 files, 0 offenses. All files have frozen_string_literal. No anti-patterns detected.

  • Zero behavior change: Default config (download_to_active_storage = false) means no jobs enqueued, no Active Storage calls, no content rewriting. Existing pipelines unaffected.

Deviations

None. Both plans executed as specified. All must_haves verified. All artifacts present and tested.

Next Steps

Phase 5 complete. REQ-24 fully satisfied. Engine now supports configurable image downloads to Active Storage with graceful fallback on all failure modes.