veraticus / dynamoid Goto Github PK

Against the new API set (2012-08-10 - fake_dynamo 0.2+), Dynamoid cannot create tables:

$ rake
[snip]
/usr/local/opt/rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/aws-sdk-1.8.0/lib/aws/core/client.rb:318:in `return_or_raise': Validation error detected: KeySchema must be a Array (AWS::DynamoDB::Errors::ValidationException)
    from /usr/local/opt/rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/gems/aws-sdk-1.8.0/lib/aws/core/client.rb:419:in `client_request'

This is because it is using the old version of the CreateTable API:

operation CreateTable
data
{"TableName"=>"dynamoid_tests_index_user_names",
 "KeySchema"=>
  {"HashKeyElement"=>{"AttributeName"=>"id", "AttributeType"=>"S"}},
 "ProvisionedThroughput"=>{"ReadCapacityUnits"=>100, "WriteCapacityUnits"=>20}}
response
{"__type"=>"com.amazon.coral.validate#ValidationException",
 "message"=>"Validation error detected: KeySchema must be a Array"}

The newer API spec expects the KeySchema to be a list of hashes with key types on them.

Is there anything in progress to update Dynamoid for the 2012 API? Should there be a note showing which version of the API this library requires, if there are significant breaking changes in the API?

I'll help do some research, and start contributing fixes to a branch. I just wanted to bring it up while I work on it, and solicit any feedback as to how it could be approached. Thanks!

Are there plans to support associations using join tables (like ActiveRecord's has and belongs to many)? For any large system, it doesn't appear to be feasible to use Dynamoid currently because the max record size of 64kb will quickly be filled up by storing the set of the join IDs in the row.

I'm trying to work with a table with four fields, similar to the example below:

class Logging
    include Dynamoid::Document
    table :name => :api_calls
    range :stamp, :integer
    field :appId, :integer
    field :function
    field :stamp, :integer
    field :version, :integer
    index :appId

When I use the table without the index directive, everything works ok (but running a query against appId uses scanning, even if I create the table with the index in AWS console beforehand).

When running it with the index directive, Dynamoid creates a table called index_api_calls_appIds and the insert fails with ArgumentError: a range key value is required for this table.

What am I doing wrong? I'm running Rails 3.2.13 and aws-sdk 1.11.1 (same problem with 1.8.6) and the latest official rubygem.

Here is an rspec test demonstrating the issue

    address = Address.create(:city => 'Seattle', :deliverable => true)
    found   = Address.find(address.id)
    found.deliverable.should eq true

    # PASSES in head
    found   = Address.where(:deliverable => 't').all
    found.should eq [address]

    # FAILS
    found   = Address.where(:deliverable => true).all
    found.should eq [address]

    # PASSES in head
    found   = Address.where(:updated_at => address.updated_at.to_time.to_f).all
    found.should eq [address]

    # FAILS
    found   = Address.where(:updated_at => address.updated_at).all
    found.should eq [address]

Dynamoid::Chain.where is not transforming the values into their dumped versions. This makes for pretty ugly queries. Instead, it could call a dump_field in Dynamoid::Persistence, however it is private at the moment: https://github.com/Veraticus/Dynamoid/blob/master/lib/dynamoid/persistence.rb#L222

You give an example:

Address.limit(5).start(address)

To get this to work, I needed to do something like this:

Address.start(address).limit(5)

I was surprised to see an array type since DynamoDB only supports sets, but I figured you must be serializing the array to JSON and storing it as a string to get around that. Instead it's just treated as a set, which is broken! It's a cool idea to support an array, but it'd be better to just call it a set if it is going to remove uniques and probably futz with the order too

  when :set, :array
    if value.is_a?(Set) || value.is_a?(Array)
      value
    else
      Set[value]
    end

I have the following models:

class User
 include Dynamoid::Document
 table :name => :users, :key => :id , :range_key => :created_at,      :read_capacity => 10, :write_capacity => 5

  field :imgurl
  field :username
  field :email 
  field :showemail #boolean
  field :about
  field :user_ids

  has_many :star

end

class Star
  include Dynamoid::Document
 table :name => :stars, :key => :id , :range_key => :created_at,      :read_capacity => 10, :write_capacity => 5
 field :title
 field :abstract
 field :stop
 field :think
 field :act
 field :review

 belongs_to :user
 end

In the a controller I have the following method:

def view
   @user = User.find_by_id(params[:id])#session[:id])

begin
  @stars = @user.star.all
rescue
  @stars = nil
end
 end

When the user has one star the request proceeds, when the user has more than one star the request returns the following error:

dynamo db Completed 500 Internal Server Error in ..

when I changed the gem version to:

  gem "dynamoid", :git => 'git://github.com/indykish/Dynamoid.git'

works perfect for when the user has more than one star.

If I have the following model setup with partitioning turned on:

class Table
  include Dynamoid::Document
  table :name => :table
  field :field1
  field :start_time, :datetime

  index :field1, :range_key => :start_time
end

Creates records fine. But when I try to Query the table with something like
Table.where(:field1 => "123", "start_time.lt" => Time.now)

This QUERY doesn't take into account the partitioning.

This makes sense as we can't really QUERY a set of partitioned ids, as dynamoDB needs a distinct HASH_KEY and RANGE_KEY to query with. But was wondering if you intended partitioning to work with indexes?

I am currently trying to find the best way to setup a schema where I can query on 2 fields and a datetime range key in one shot.

Our current solution involves combining 2 fields as the id as seen below:

class Table  
  include Dynamoid::Document
  table :name => :table
  range :start_time, :datetime 
  field :field1
  field :field2
  field :field3
end

and inserting like

Table.new( :id => field1 + '.' + field2, , :start_time => date, :field1 => field1, :field2 => field2, :field3 => field3)

Unfortunately, this doesn't spread the hash_keys out for uniform workload. but works!

Any insight would be fantastic.

If we have the following table:

class Website
   include Dynamoid::Document

   range :created_at, :integer
   field :subdomain, :string

   index :subdomain
end

The created index table will have the RANGE key set to created_at, but when a new item is created, the range key is not set.

Website.create(:created_at => Time.now.to_i, :subdomain => 'test') will fail.

I think such cases should have RANGE set to the original range.

Hi, I successfully integrated the gem in my app, I can store and retrieve data from dynamoDB, no issue on it. But when i log into the console.aws.amazon.com/dynamodb i was aspecting to find the hash that i saved from my app but I still have the wizard used to create a new table and so on...it is normal?
Thank you

It would be great if dynamoid could support loose model, and valorize attributes based on the columns it finds in the dynamodb field

As the range key paragraph in the README suggests, I'm searching for past documents:

LogEntry.where("created_at.lt" => (DateTime.now - 1.week)).all

What I get in return is:

AWS::DynamoDB::Errors::ValidationException: One or more parameter values were invalid: An AttributeValue may not contain an empty string.
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/client.rb:273:in `return_or_raise'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/client.rb:372:in `client_request'
from (eval):3:in `query'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/dynamo_db/item_collection.rb:788:in `_each_item'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/collection/limitable.rb:57:in `each_batch'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/core/collection.rb:64:in `each'
from ~/.rvm/gems/ruby-1.9.2-p180/gems/aws-sdk-1.5.2/lib/aws/dynamo_db/item_collection.rb:496:in `each'
from ~/.rvm/gems/ruby-1.9.2-p180/bundler/gems/Dynamoid-61de8e9ce41e/lib/dynamoid/adapter/aws_sdk.rb:180:in `each

My Dynamoid document looks like this:

class LogEntry
  include Dynamoid::Document
  field :query
  index :created_at, range: true
end

The table was created implicitly by Dynamoid upon first access.

Is something wrong with the gem or am I missing something?

Maybe I am wrong, but shouldn't where (:id => '') use QUERY instead of SCAN when id is hash key?

/usr/lib/ruby/gems/1.8/gems/dynamoid-0.4.1/lib/dynamoid/adapter/aws_sdk.rb:66:in `create_table': uninitialized constant Dynamoid::Adapter::AwsSdk::Dynamite (NameError)

I am getting the above error when trying to use the gem. Anyone know why it is happening?

Hi,

It would be really nice to have a uniqueness validator.

validates :name, :uniqueness => true

That's all.

Thanks,
Mateusz

Hi, I have a model called StatDisplayDynamo

class StatDisplayDynamo
  include Dynamoid::Document

  table :name => :stat_display_dynamo, :key => :id, :read_capacity => 400, :write_capacity => 400

  field :campaign_id, :integer
  field :ip_address
  field :ua_raw
  field :ua_name
  field :ua_version
  field :ua_engine
  field :ua_os
  field :ua_engine_version

  index :created_at, :range => true

  belongs_to :company
end

with one entry in

StatDisplayDynamo.all
=> [#<StatDisplayDynamo:0x00000104dc2c38 @new_record=false, @attributes={:created_at=>Tue, 06 Aug 2013 20:55:21 +0200, :updated_at=>Tue, 06 Aug 2013 20:55:21 +0200, :id=>"b9b769e7-bdb7-4225-8215-842caaa1652c", :campaign_id=>114, :ip_address=>"127.0.0.1", :ua_raw=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:22.0) Gecko/20100101 Firefox/22.0", :ua_name=>"Unknown", :ua_version=>nil, :ua_engine=>"gecko", :ua_os=>"OS X 10.8", :ua_engine_version=>"20100101", :company_ids=>nil}, @Associations={}, @changed_attributes={}>]

but if I query in this way

StatDisplayDynamo.where("created_at.gt" => DateTime.now-2.hours).all

when

DateTime.now-2.hours => Tue, 06 Aug 2013 19:34:24 +0200

I get no result. Where am i wrong?

DescribeTable return ItemCount without a full scan.
For pagination, or other stuff, it will be great to add a count method :

dynamodb = AWS::DynamoDB.new
dynamodb.client.describe_table(table_name: TABLE_NAME)["Table"]["ItemCount"]

Thanks

Is there any facility for Dynamoid to throw ActiveRecord::RecordNotFound errors? Am I missing it somewhere?

This functionality would help me clean up my controller code and make this a much cleaner ORM.

How do I pass query options in a find_all_by_* query? Seems they are not passing through to the underlying query

Comment.find_all_by_user_id_and_blog_id "1234", "1", limit: 1, scan_index_forward: true
Comment.find_all_by_user_id "1234", limit: 1, scan_index_forward: true

If I use query directly, it works as expected:

    query_options {hash_value: "12345", scan_index_forward: true, limit: 1}
    hash_attrs = Dynamoid::Adapter.adapter.query Comment.table_name, query_options
    hash_attrs.map{|attrs| self.from_database attrs}

Hello,

I just checked out Dynamoid tip and rand the rspecs, the AWS SDK does not like the scan_index_forward option introduced in commit 67b4093. Before this commit, all tests pass, afterwards, many tests fail.

Hey there,

We're looking into using Unicorn's 'preload_app' option, which speeds up app startup and reduces memory usage by spawning a single instance of your application, and then forking that several times for each worker.

The only downside is that you need to make sure you re-initiate all external connections in an 'after_fork' block. We've set up ours to do the following:

after_fork do |server, worker|
  Dynamoid::Adapter.reconnect!
end

Unfortunately, this doesn't seem to work as expected, as running under Unicorn with 'preload_app true' we sometimes get responses back that appear to be for a different request, which causes problems. If we set 'preload_app false' this issue goes away.

We're not 100% certain the issue is in Dynamoid itself, but figured you'd be a good person to ask :D

If you have any advice for using Dynamoid within an environment such as Unicorn preload, or Passenger when set to spawning mode, I welcome it!

Thanks!
Ryan

I have two models (ShoppingCart & Item) where ShoppingCart has_many :items and Item belongs_to :shopping_cart. Without validations on the Item everything works beautifully. But as soon as I add a validates_presence_of :quantity on the Item model it gives me the following error:

ArgumentError (unsupported attribute type NilClass in value for attribute items_ids):
  app/controllers/item_controller.rb:17:in `create'

Removing the validates_presence_of makes it work correctly again.

You have to use find_all instead of find for tables with range keys because in dynamoid find destructively calls flatten and unique on the array of ids. Why is this a problem? Because find calls find_all, which calls read, which eventually calls AWS::DynamoDB::BatchGet, which expects arguments in this form:

ids = [ ['hash_key1', 'range_key1'], ['hash_key2', 'range_key2'] ]

flattened becomes

ids.flatten.uniq
=> ["hash_key1", "range_key1", "hash_key2", "range_key2"]

which is wrong and fails. The solution is to change Dynamoid\lib\dynamoid\finders.rb#26 to only flatten the ids when the table has no range key, for example:

  ids = Array(self.range_key ? ids.flatten.uniq : ids.uniq)

I may be missing something, but there is a problem with the default configuration: the default value in lib/dynamoid/config.rb sets Dynamoid::Config.adapter to be 'aws-sdk', but this causes a problem with how the reconnect! method is currently implemented:

# lib\dynamoid\adapter.rb
module Dynamoid
  module Adapter
    def reconnect!
      require "dynamoid/adapter/#{Dynamoid::Config.adapter}" unless Dynamoid::Adapter.const_defined?(Dynamoid::Config.adapter.camelcase)
      @adapter = Dynamoid::Adapter.const_get(Dynamoid::Config.adapter.camelcase)
      @adapter.connect! if @adapter.respond_to?(:connect!)
      self.tables = benchmark('Cache Tables') {list_tables}
    end
  end
end

require "dynamoid/adapter/aws-sdk" will fail, the file name is actually aws_sdk. Also on the 2nd line Dynamoid::Config.adapter.camelcase gives 'Aws-sdk' which I think is supposed to be Dynamoid::Adapter::AwsSdk.

I found this method to be too complex and monkey patched it for my application to something more simple:

# lib\dynamoid\adapter.rb
module Dynamoid
  module Adapter
    def reconnect!
      @adapter = Dynamoid::Adapter::AwsSdk
      @adapter.connect!
      self.tables = benchmark('Cache Tables') {list_tables}
    end
  end
end

For a more general solution I would say, if you want to use the adapter in reconnect, leave the responsibility of the caller to require the file. I did not test this, but perhaps something like might do the trick:

# initializer
require 'some/custom_adapter'
Dynamoid.configure do |config|
  config.adapter = Some::CustomAdapter
end

# lib\dynamoid\adapter.rb
module Dynamoid
  module Adapter
    def reconnect!
      @adapter = Dynamoid::Config.adapter
      @adapter.connect!
      self.tables = benchmark('Cache Tables') {list_tables}
    end
  end
end

Even if you do not push the responsibility of requiring it to the caller, it should work with the defaults.

Followed all the step but I'm getting following error

/.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:251:in `require': cannot load such file -- dynamoid/adapter/aws (LoadError)

from /.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:251:in `block in require'
from /.rvm/gems/ruby-1.9.3-p392/gems/activesupport-3.2.13.rc1/lib/active_support/dependencies.rb:236:in `load_dependency'

Is it possible to create a table with a range key? Specifically, I'm looking to make an index on created_at. If I specify it as an index, then it creates a second table which is less than ideal. Shouldn't table and field take the :range option?

What do you think about adding some generators like mongoid has? If you think they would fit in, I am willing to implement them myself.

Seems that the change for 0.3.2 has caused the above error

Dynamite? Where does it come from ? How is this possible ? I've basically only recreated the sample in the manual..

:001 > User.find(:all)
NameError: uninitialized constant Dynamoid::Adapter::AwsSdk::Dynamite
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter/aws_sdk.rb:66:in create_table' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:126:inblock (3 levels) in module:Adapter'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:39:in benchmark' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/adapter.rb:126:inblock (2 levels) in module:Adapter'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/persistence.rb:50:in create_table' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:45:inblock in create_indexes'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:42:in each' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:42:increate_indexes'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/dynamoid-0.4.1/lib/dynamoid/indexes.rb:28:in index' from /Users/kjetil/Dropbox/Development/railsprojects/DynTest/app/models/user.rb:11:inclass:User'
from /Users/kjetil/Dropbox/Development/railsprojects/DynTest/app/models/user.rb:1:in <top (required)>' from (irb):1 from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands/console.rb:45:instart'
from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands/console.rb:8:in start' from /Users/kjetil/.rvm/gems/ruby-1.9.2-p290@rails31-zen/gems/railties-3.1.0/lib/rails/commands.rb:40:in<top (required)>'
from script/rails:6:in require' from script/rails:6:in

I had a question about using Dynamoid to create a new table. I had a new table when I tried to start using it, but when I first tried to read from it, I got a ResourceNotFoundException. I couldn't figure out how to make Dynamoid create the table for me, so I had to create it manually, and hope that I got the table and key name correct (I did, yay!).

Is there a way to make Dynamoid create the table on my behalf? The documentation implies yes, but it doesn't say how.

Thanks a lot!

I am using dynamoid which seems to work great, but how are people using dynamoid for testing? I want my rspec tests to be able to make nice calls, but not actually hit dynamodb.

Hello guys I wrote a small program that basically just configures a Dynamoid object the code is the following:

 require 'dynamoid'

Dynamoid.configure do |config|
config.adapter = 'aws_sdk' # This adapter establishes a connection to the DynamoDB servers using Amazon's own AWS gem.
config.access_key = 'lalal' # If connecting to DynamoDB, your access key is required.
config.secret_key = 'opaaaa' # So is your secret key.
config.endpoint = 'dynamodb.eu-west-1.amazonaws.com' # Set the regional endpoint for DynamoDB.
config.namespace = "dynamoid_app_development" # To namespace tables created by Dynamoid from other tables you might have.
config.warn_on_scan = true # Output a warning to the logger when you perform a scan rather than a query on a table.
config.partitioning = true # Spread writes randomly across the database. See "partitioning" below for more.
config.partition_size = 200  # Determine the key space size that writes are randomly spread across.
config.read_capacity = 100 # Read capacity for your tables
config.write_capacity = 20 # Write capacity for your tables
end

just running this I get the following error:

wrong constant name Aws-sdk (NameError)

P.S.:
If you could direct me to a more complete tutorial than your getting started guide I would be delighted

With these classes

# app/models/vehicle.rb
class Vehicle
  include Dynamoid::Document
  table key: :vehicle_id
  field :vehicle_id
  field :type
end

# app/models/car.rb
class Car < Vehicle
end

# app/models/boat.rb
class Boat < Vehicle
end

Car.all and Car.find(id) return the same thing Vehicle.all and Vehicle.find(id), which include Boat objects.

Hi,

I'd like to use DDB's standard hash+range key on a table without using Dynamoid's "index" function. Is this possible?

Sometimes I get different data (same object saved at different points in time) after reloading:

1.9.3p194 :009 > u = User.first ; u.first_name
=> "Aa"
1.9.3p194 :010 > u.reload ; u.first_name
=> "Aaron"

In the log, it shows that User.first uses SCAN and u.reload uses BATCH GET ITEM. I was thinking that might be a clue.

I also suspect I may not be using partitioning correctly. If I just turn it off, will it make everything simpler?

The u.reload version is the correct (latest) one. This has been maddening when trying to do a password change, because the password change completes successfully, but then when I try to log in with the new password, the system compares it to some older version of the password hash, and it fails.

Lets assume that we have a table with the following structure.

class Website
    include Dynamoid::Document

    range :page, :integer
    field :subdomain, :string
    index :subdomain

end

If we do Website.find_by_subdomain('test') we will get the following error: "ArgumentError: a range key value is required for this table". This happens because Dynamoid uses table.items.at (https://github.com/Veraticus/Dynamoid/blob/master/lib/dynamoid/adapter/aws_sdk.rb#L162 ) which would expect to have a range key because the at method returns only one item.

A better approach would be to use the query method from the client which allows to search for an item by HASH even if it has a RANGE defined.

I have tried all kind of ways to do such a query (which is possible in the AWS console) but I haven't got nowhere so far. Please let me know what you think.

range_key for indexes is set to :number all the time. Why not allow other type of RANGE for indexes? An index basically creates another table which uses the index as hash primary key. I think the range should be allowed to be at least datetime too.

Specific issue: add performance benchmarks for Dynamoid::Config.partitioning? (provide a link to results in the readme).

Do you guys have performance benchmarks for Dynamoid::Config.partitioning? ? It seems at a glance that it does give a benefit of balanced writes (up to a 200x data volume), but it also requires 200x read capacity for any object that has been updated sufficient number of times.

Do you actively use this feature at scale? Is it actively supported or are you planning to pull it out?

Hi,

How we can introduce the new LSI (Local Secondary Indexes) in Dynamoid ? It can be really useful in order to avoid create a table for index.

Is that in the roadmap ?

When running the server with Dynamoid, the following warning is generated:

/Library/Ruby/Gems/1.8/gems/dynamoid-0.3.2/lib/dynamoid/persistence.rb:181: warning: found = in conditional, should be ==

Thanks

It looks like there is an error on line 44 of Persistence. I continually get:

undefined method `symbolize_keys!' for {"name"=>"se", "email_addr"=>"[email protected]"}:ActiveSupport::HashWithIndifferentAccess

When doing a new or create.

Index work fine only when the HASH KEY is id.

In my opinion, indexes should not map obj.id but [HASH_KEY, RANGE_VALUE] and use it as an identifier for the row the index is added to. This will make it easy to retrieve the item from the original table this way.

Hey guys, how is it going?

It is the first time I'm using Dynamoid and it looks like a found I bug. I'm not sure if I'm doing something wrong, but here is it goes:

The following model works fine:

class Dytest

  include Dynamoid::Document

  table name: "#{Rails.env}_preferences"

  field :foo
  field :bar

end

but if I set a custom hash key it breaks the query:

class Dytest

  include Dynamoid::Document

  table name: "#{Rails.env}_preferences", key: :entity_uuid

  field :foo
  field :bar

end

d = Dytest.new(foo: '42', bar: 'monkey')
d.save

Dytest.all #here it blows up

Backtrace:

NoMethodError: undefined method `split' for nil:NilClass
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:155:in `block (2 levels) in result_for_partition'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:153:in `each'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:153:in `block in result_for_partition'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:152:in `tap'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:152:in `result_for_partition'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/adapter.rb:115:in `scan'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:148:in `records_without_index'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:90:in `records'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria/chain.rb:46:in `all'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/dynamoid-0.5.0/lib/dynamoid/criteria.rb:20:in `block (2 levels) in <module:ClassMethods>'
    from (irb):4
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands/console.rb:45:in `start'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands/console.rb:8:in `start'
    from /Users/esdrasmayrink/.rvm/gems/ruby-1.9.2-p320@organizze/gems/railties-3.1.4/lib/rails/commands.rb:40:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'

It seems that this method is where it blows up.

# It seems that result[:id] is nil because I defined a custom hash key, am I right?
def result_for_partition(results)
  {}.tap do |hash|
    Array(results).each do |result|
      next if result.nil?
      id = result[:id].split('.').first
      if !hash[id] || (result[:updated_at] > hash[id][:updated_at])
        result[:id] = id
        hash[id] = result
      end
    end
  end.values
end

Please let me know if there is anything I can do to help.

Cheers, Esdras.

Hi,

I'd like to be able to create the dynamoid tables without actually saving a record. I can't seem to find this in the documentation somewhere. Is it possible?

Model.exists? returns true all the time.

The #{attribute_name}_changed? and similar methods from ActiveModel::Dirty do not work except for the created_at attribute if you're using a version of ActiveModel older that 3.1.0.

I'm not sure if it's realistic to make it do that or not, but it's a bug at least to not require a newer version of ActiveModel.

I believe the endpoint needs to be quoted in the Readme:

:dynamo_db_endpoint => dynamodb.ap-southeast-1.amazonaws.com

I think a set should be mapped differently as an index. If we have an object Book and we want to add Tags (as a set ["SF", "Drama"]) to it and it as index, then in the index table there should be created 2 items, one for each entry in the tags set. The first would have all the books with the "SF" tag and the other one all the books that have the "Drama" tag.

This is test result of my application .
Amazon DynamoDB does not allow ":" character in table name.
Dynamoid is trying to create a "rspec_index_pastehub::entry_usernames" table.

$ rake test_e
time ruby -I ./lib `which rspec` -b  -t entries ./test/aws_spec.rb            -r ./test/rspec_formatter_for_emacs.rb -f CustomFormatter
Amazon DynamoDB access_key_id:     XXXXXXXXXXXXXXXXXXXX
Amazon DynamoDB secret_access_key: YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:3 warning: already initialized constant version.rb:3
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:4 warning: already initialized constant VERSION_ARRAY
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:5 warning: already initialized constant VERSION_MAJOR
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:6 warning: already initialized constant VERSION_MINOR
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/version.rb:7 warning: already initialized constant VERSION_BUILD
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:98 warning: already initialized constant NaN
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:100 warning: already initialized constant Infinity
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:102 warning: already initialized constant MinusInfinity
/usr/local/stow/ruby-1.7.0.preview1-jruby/lib/ruby/gems/shared/gems/json-1.6.5-java/lib/json/common.rb:121 warning: already initialized constant UnparserError
(5472.0 ms) LIST TABLES - [[]]
(5485.0 ms) CACHE TABLESCreating rspec_index_pastehub::entry_usernames table. This could take a while.
#<Class:0x28c61629>: 1 validation error detected: Value 'rspec_index_pastehub::entry_usernames' at 'tableName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+

I require to batch a collection of items together for batchPut call. Can't find an easy way to do this using Dynamoid. Am I missing something?