select_all loads all IDs in memory about randumb HOT 3 CLOSED

Yes this is certainly an issue, but I'm not sure the best way around it.

I suppose in the cases when only 1 record is desired, we might first do a count of the records that match the criteria, then do the query again with limit 1 and a random offset. Then maybe keep doing this for each additional record? This might be more performant when small numbers of records are requested.

Or perhaps the more common way of doing random ( order by rand() in the DB ), might be more performant with large record sets.

In either case, I'll do some benchmarking with the above to ideas and compare them to the existing algorithm with super large record sets.

Any suggestions or thoughts?

from randumb.

I changed my DB schema to have much smaller amount of records in need random access to. So order by random() works fast enough.

Depending on the query you could also do

• m = select max(id) from table"
• select * from table where id >= rand(m + 1)

(if you have additional where conditions then this might not be evenly distributed)

There is no ideal solution to this, i guess. Perhaps just describe how the gem is doing it (and alternatives) so one does not have to read the source code.

from randumb.

After some testing, it turns out that it is almost always faster to just order by random at the database level. I just updated the gem to do this, should work for mysql, sqlite, and postgres. So no more ids in memory :)

from randumb.