When I try to bulk import via:

Mailbox::Email.import! dataset: Mailbox::Email.where { ...conditions }

It does the job but I get following warning. Elasticsearch seems deprecated this feature and probably will remove on upcoming versions.

warning: 299 Elasticsearch-7.10.0-51e9d6f22758d0374a0f3f5c6e8f3a7997850f96 "[types removal] Specifying types in bulk requests is deprecated."

What is/are our option(s) in here?

Hey I have another question, why here you are using sync as default type instead of _doc?

Since the request will be PUT elasticsearch/index/sync/id instead of PUT elasticsearch/index/_doc/id

Hey, if i'm using the plugin in this way:

class Document < Sequel::Model
  plugin :elasticsearch
end

How could I specify the config for elasticsearch? such like username/password?
I saw you wrote

Sequel::Model.plugin :elasticsearch,
  elasticsearch: { log: true }, # Options to pass the the Elasticsearch ruby client
  index: 'all-my-data', # The index in which the data should be stored. Defaults to the table name associated with the model
  type: 'is-mine' # The type in which the data should be stored.

but where should I put this？

I update models in batch like this:

Email.where(user_id: user.id, guid: guids).update(...)

So, hooks are not called. Is there a way to invoke and update affected ids/guids (which is available on es)? For example;

Email.update_es!(dataset: Email.where(guid: guids))

This will force update all emails based on dataset

I've found that when I load records from Sequel into ElasticSearch, I always have to remap the DateTime columns with #iso_8601 for it to get picked up as a timestamp in ES.

I use SQL Server and Sequel/TinyTds and I'm not sure if folks on Postgres and others experience the same issues.

For example, when running a_model_object.db_schema.map{|col, v| [col, v[:db_type] ]}, I can see each of the data types, and I know that datetime columns need special attention in ES before I send the data.

While learning about ES, I've been loading my data using the bulk method with methods that look like this:

def format_record values
  values[:started_at] = values[:started_at] && values[:started_at].iso8601
  values[:completed_at] = values[:completed_at] && values[:completed_at].iso8601
  [
    { index: { _index: 'jobs', _type: 'job', _id: values[:id] } },
    values
  ]
end

In my research, it seems to be an issue with the value formatted from Ruby's Time.new.to_s

The above example could obviously be refactored to be a bit more dynamic, perhaps something along the lines of values.each{|k,v| values[k] = v.iso_8601 if v.is_a? Time}

Trying to force the mapping explicitly in elasticsearch as date yields messages like:

"Invalid format: \"2018-02-06 04:38:47 -0600\" is malformed at \" 04:38:47 -0600\""

Essentially, as far as I can tell, all DateTime columns / ruby Time objects should be converted to #iso_8601 before being loaded for es to see it correctly. But I'm very new to ES, so I may be missing something.

Hello,

I'm working on a parity to ElasticSearch-Rails, which adds an overridable method to ActiveRecord::Model::InstanceVariables for the indexed values (#as_indexed_json).

So, for Sequel::Elasticsearch, I'm thinking of adding InstanceMethods#indexed_values to lib/sequel/plugins/elasticsearch.rb with default implementation simply return Sequel::Model#values.
Then modify #index_document to use #indexed_values.

I'll be working on this sometime in the next couple of weeks. Just to let you know this issue is on my radar. I"ve forked the project for my dev.

Kindest Regards,
Peter Fitzgibbons

Hello,
I am writing a worker-component with sequel-elasticsearch whose index must also be read by an equivalent ActiveRecord model with Elasticsearch::Model plugin.
I am not seeing the same results between the two.

I am seeing this for ElasticSearch::Model index name:

# class Forum < ActiveRecord::Base
#       include Elasticsearch::Model
#      index_name "throtl-admin-forum-#{Rails.env}"
Forum.__elasticsearch__.index_name
=> "throtl-admin-forum-development"

And this for sequel-elasticsearch index name:

#    class Forum < Sequel::Model
#      env = ENV['RAILS_ENV'] || 'development'
#      plugin :elasticsearch,
#             index: "throtl-admin-forum-#{env}".to_sym
Scraper::DB::Forum.elasticsearch_index
 => :"throtl-admin-forum-development"

So:
"throtl-admin-forum-development" # Elasticsearch::Model
vs
:"throtl-admin-forum-development" # sequel-elasticsearch

I think the issue is that the sequel-elasticsearch index name somehow has the quotes captured in the symbolized index name? I even tried to force-symbolize the key in my code above, without affect.

Could you tell me if you have any ideas how this can be done correctly?

Thanks and Kindest Regards,
Peter Fitzgibbons