blocknotes / active_storage_db Goto Github PK

An ActiveStorage service plugin to store files in a PostgreSQL / MySQL / MSSQL database (for Rails 6/7)

License: MIT License

Ruby 85.49% JavaScript 0.87% CSS 1.38% HTML 12.24% Shell 0.03%

ruby rails activestorage activestorage-service mysql postgres postgresql

active_storage_db's Introduction

Active Storage DB

An Active Storage service upload/download plugin that stores files in a PostgreSQL or MySQL database. Experimental support also for MSSQL.

Main features:

attachment data stored in a binary field (or blob);
all service methods implemented;
supports Rails 6 and 7.

Useful also with platforms like Heroku (due to their ephemeral file system).

Installation

Setup Active Storage in your Rails application
Add to your Gemfile gem 'active_storage_db' (and execute: bundle)
Install the gem migrations: bin/rails active_storage_db:install:migrations (and execute: bin/rails db:migrate)
Add to your config/routes.rb: mount ActiveStorageDB::Engine => '/active_storage_db'
Change Active Storage service in config/environments/development.rb to: config.active_storage.service = :db
Add to config/storage.yml:

db:
  service: DB

Misc

Some utility tasks are available:

# list attachments ordered by blob id desc (with limit 100):
bin/rails 'asdb:list'
# search attachments by filename (or part of it)
bin/rails 'asdb:search[some_filename]'
# download attachment by blob id (retrieved with list or search tasks) - the second argument is the destination:
bin/rails 'asdb:download[123,/tmp]'

Do you like it? Star it!

If you use this component just star it. A developer is more motivated to improve a project when there is some interest.

Or consider offering me a coffee, it's a small thing but it is greatly appreciated: about me.

Contributors

Mattia Roccoberton: author
Inspired by activestorage-database-service project

License

The gem is available as open source under the terms of the MIT License.

active_storage_db's People

Contributors

Stargazers

Watchers

Forkers

henkeinfo nerdoodev jinshen-cn collectiveidea francois

active_storage_db's Issues

PG::UndefinedFunction: ERROR: function max(uuid)

+ ./bin/rails asdb:list
rails aborted!
ActiveRecord::StatementInvalid: PG::UndefinedFunction: ERROR:  function max(uuid) does not exist
LINE 1: SELECT MAX("active_storage_blobs"."id") FROM "active_storage...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

This seems to be working well with ActionMailbox - active_storage_db_files has a single record - with a size that makes sense for an email's content + the attachment size. - and I can view the email here - http://localhost:3000/rails/conductor/action_mailbox/inbound_emails

but ./bin/rails asdb:list produces the error above.

Can I ignore this? - has this been designed with non-uuids in mind?

Going to deploy to render.com - where it will either pass or fail - but seems to work.

ERROR: null value in column "record_id" of relation "active_storage_attachments" violates not-null constraint

I am using your plugin with ActionMailbox,
I followed instructions in readme.

http://localhost:3000/rails/conductor/action_mailbox/inbound_emails/new
If I send an email in development

[2023-01-25T17:05:46.419889 #79606] DEBUG -- :   ActiveStorage::Attachment Load (2.6ms)  SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_id" = $1 AND "active_storage_attachments"."record_type" = $2 AND "active_storage_attachments"."name" = $3 LIMIT $4  [["record_id", nil], ["record_type", "ActionMailbox::InboundEmail"], ["name", "raw_email"], ["LIMIT", 1]]
D, [2023-01-25T17:05:46.431688 #79606] DEBUG -- :   ActiveStorage::Attachment Create (7.8ms)  INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"  [["name", "raw_email"], ["record_type", "ActionMailbox::InboundEmail"], ["record_id", nil], ["blob_id", "85be5cc3-6e9b-4548-9d09-313dba0b7484"], ["created_at", "2023-01-25 16:05:46.423136"]]
D, [2023-01-25T17:05:46.435306 #79606] DEBUG -- :   TRANSACTION (1.6ms)  ROLLBACK
I, [2023-01-25T17:05:46.437930 #79606]  INFO -- : Completed 500 Internal Server Error in 274ms (ActiveRecord: 70.2ms | Allocations: 50127)


F, [2023-01-25T17:05:46.440553 #79606] FATAL -- :   
ActiveRecord::NotNullViolation (PG::NotNullViolation: ERROR:  null value in column "record_id" of relation "active_storage_attachments" violates not-null constraint
DETAIL:  Failing row contains (78effb50-dd85-4101-9aed-4133cb9ccee7, raw_email, ActionMailbox::InboundEmail, null, 85be5cc3-6e9b-4548-9d09-313dba0b7484, 2023-01-25 16:05:46.423136).
):
  
activerecord (7.0.4.1) lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `exec_params'
activerecord (7.0.4.1) lib/active_record/connection_adapters/postgresql_adapter.rb:768:in `block (2 levels) in exec_no_cache'
activesupport (7.0.4.1) lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
activesupport (7.0.4.1) lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'

btw - this URL - /active_storage_db -does not seem to work.
and
r 'asdb:list'

produces

ActiveRecord::StatementInvalid: PG::UndefinedFunction: ERROR:  function max(uuid) does not exist
LINE 1: SELECT MAX("active_storage_blobs"."id") FROM "active_storage...
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

Support for Rails 5.2

Hey guys,

at first good to see things keep moving using the db as active storage backend after having a look on alternative projects. This looks like to the best version around.
Just one question, is there a reason why the gemspec requires rails 6.0? I'm just playing around a bit and it seems to work with 5.2, too. After setting rails and active storage to 5.2 there were no failures for the mysql specs. I haven't tested postgres. I will now go on and check if there are issues arising with 5.2 in development.

SQL Server/TinyTDS Error for OCTET_LENGTH

Hello, and thank you for creating this gem - it has been very useful.

I am using ActiveStorageDB with SQL Server via TinyTDS. I'm aware that ActiveStorageDB currently specifies support for Postgres and MySQL only, but I figured I'd ask if SQL Server could be considered for future support, or if you have any insight into a possible workaround for my specific issue.

I am encountering errors when attempting to use ActiveStorage variants with ActiveStorageDB. I got an error related to AnalyzeJob after attaching images via ActiveStorage, but displaying images was ok. However, the same error occurs when attempting to display variants. The error displayed is:

ActionView::Template::Error (TinyTds::Error: 'OCTET_LENGTH' is not a recognized built-in function name.)

The SQL statement causing this issue is:

ActiveStorageDB::File Load (45.7ms)  EXEC sp_executesql N'SELECT OCTET_LENGTH(data) AS size FROM [active_storage_db_files] WHERE [active_storage_db_files].[ref] = @0 ORDER BY [active_storage_db_files].[id] ASC OFFSET 0 ROWS FETCH NEXT @1 ROWS ONLY', N'@0 nvarchar(4000), @1 int', @0 = N'wbcflekvut7wmxukoxa535n8hwfg', @1 = 1  [["ref", nil], ["LIMIT", nil]]

I think SQL Server has either LEN or DATALENGTH, though without deep diving into the code I'm not sure which would be better to use, if any, or if it could be easily patched in.

I understand that SQL Server is not officially supported but any guidance on how to fix this would be appreciated.

Filtering of the log messages?

I notice the message that is logged when the blob is written to the DB is pretty massive... Is there any way to filter it out?

Here is what I see (with the data all removed except for the first six bytes...

ActiveStorageDB::File Create (2.8ms)  INSERT INTO `active_storage_db_files` (`ref`, `data`, `created_at`) VALUES ('7313t5zetar6ndasua7kpqb7q651', x'789ced

with Mysql, do we have any sense of how large you can make a file? I am storing large JSON files (like up to 100 mb) for background processing.. goign to play with compressing them before uploading!

What would it take to use under SQLite3?

I'd be interested in running this gem with SQLite3 instead of PG. Backing up the database in a single file would be so very interesting!

I tried setting up Appraisal (and removing MySQL and MSSQL as I don't have that installed), but when I ran, I got the following:

$ bundle exec appraisal rails-7_1-sqlite3 rspec
>> BUNDLE_GEMFILE=/Users/francois/Projects/scoutges/active_storage_db/gemfiles/rails_7_1_sqlite3.gemfile bundle exec rspec

An error occurred while loading rails_helper.
Failure/Error: require File.expand_path("dummy#{app_ver}/config/environment.rb", __dir__)

LoadError:
  cannot load such file -- /Users/francois/Projects/scoutges/active_storage_db/spec/dummy/config/environment.rb
# ./spec/rails_helper.rb:37:in `<top (required)>'
No examples found.

Finished in 0.00003 seconds (files took 0.21598 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

I suspect the environment isn't correctly setup.

I would be interested in making this gem work. What would it take?

blocknotes / active_storage_db Goto Github PK

active_storage_db's Introduction

Active Storage DB

Installation

Misc

Do you like it? Star it!

Contributors

License

active_storage_db's People

Contributors

Stargazers

Watchers

Forkers

active_storage_db's Issues

Recommend Projects

Recommend Topics

Recommend Org