feat: Support SQLite as Offline Store about feast HOT 7 OPEN

Do we? pretty sure Hao is referring to sqlite offline store engine. We don't currently have that, sqlite is only used on the online side.

from feast.

Yep, i'm talking about sqlite as data source for offline store. Actually it's bit confusing that data source is in some way equals to offline store. I feels it should be a abstraction that can be used for both offline store and online store? @franciscojavierarceo @tokoko

from feast.

We already support this. Do you have something more specific you want?

from feast.

Yeah that makes sense! And I do agree data source is weird to be limited exclusively to offline store.

from feast.

@franciscojavierarceo @HaoXuAI Before we go too far here, let me dig deeper in the use case. We already support duckdb which can be made to work even for local testing w/o extra infrastructure similar to how sqlite works. What do we gain by adding sqlite?

from feast.

@tokoko

1. Feast is meant to be flexible, so if someone chooses to contribute an online or offline store our bias should be to accept the implementation and make sure it works before merging.

2. SQLite is one of the most popular local databases and it comes out of the box without any necessary installations, so supporting it seems rational for local development.

As LLMs become increasingly local, (2) plays a bigger role.

Separately, I do agree DuckDB seems like a tool we should certainly advise users towards.

from feast.

Yeah I think it's still worth adding the SQLite, probably the most simple db in python world :)

from feast.