Git Product home page Git Product logo

dolthub / dolt Goto Github PK

View Code? Open in Web Editor NEW
17.5K 113.0 498.0 141.55 MB

Dolt – Git for Data

License: Apache License 2.0

Shell 14.64% Perl 0.02% Go 84.70% Makefile 0.02% Python 0.07% Java 0.06% JavaScript 0.28% C 0.03% C# 0.05% CMake 0.01% C++ 0.02% Ruby 0.02% Elixir 0.01% R 0.03% Batchfile 0.01% Dockerfile 0.01% TypeScript 0.02% Rust 0.01% PHP 0.01%
database data-version-control data-versioning sql git git-for-data mysql golang database-version-control version-controlled-database

dolt's Introduction

Dolt is Git for Data!

Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a Git repository.

Connect to Dolt just like any MySQL database to read or modify schema and data. Version control functionality is exposed in SQL via system tables, functions, and procedures.

Or, use the Git-like command line interface to import CSV files, commit your changes, push them to a remote, or merge your teammate's changes. All the commands you know for Git work exactly the same for Dolt.

Git versions files. Dolt versions tables. It's like Git and MySQL had a baby.

We also built DoltHub, a place to share Dolt databases. We host public data for free. If you want to host your own version of DoltHub, we have DoltLab. If you want us to run a Dolt server for you, we have Hosted Dolt. If you are looking for a Postgres version of Dolt, we built DoltgreSQL. Warning, it's early Alpha. Dolt is production-ready.

Join us on Discord to say hi and ask questions, or check out our roadmap to see what we're building next.

Video Introduction

Dolt Explainer Video

What's it for?

Lots of things! Dolt is a generally useful tool with countless applications. But if you want some ideas, here's how people are using it so far.

Dolt can be set up as a replica of your existing MySQL or MariaDB database using standard MySQL binlog replication. Every write becomes a Dolt commit. This is a great way to get the version control benefits of Dolt and keep an existing MySQL or MariaDB database.

Dolt CLI

The dolt CLI has the same commands as git, with some extras.

$ dolt
Valid commands for dolt are
                init - Create an empty Dolt data repository.
              status - Show the working tree status.
                 add - Add table changes to the list of staged table changes.
                diff - Diff a table.
               reset - Remove table changes from the list of staged table changes.
               clean - Remove untracked tables from working set.
              commit - Record changes to the repository.
                 sql - Run a SQL query against tables in repository.
          sql-server - Start a MySQL-compatible server.
          sql-client - Starts a built-in MySQL client.
                 log - Show commit logs.
              branch - Create, list, edit, delete branches.
            checkout - Checkout a branch or overwrite a table from HEAD.
               merge - Merge a branch.
           conflicts - Commands for viewing and resolving merge conflicts.
         cherry-pick - Apply the changes introduced by an existing commit.
              revert - Undo the changes introduced in a commit.
               clone - Clone from a remote data repository.
               fetch - Update the database from a remote data repository.
                pull - Fetch from a dolt remote data repository and merge.
                push - Push to a dolt remote.
              config - Dolt configuration.
              remote - Manage set of tracked repositories.
              backup - Manage a set of server backups.
               login - Login to a dolt remote host.
               creds - Commands for managing credentials.
                  ls - List tables in the working set.
              schema - Commands for showing and importing table schemas.
               table - Commands for copying, renaming, deleting, and exporting tables.
                 tag - Create, list, delete tags.
               blame - Show what revision and author last modified each row of a table.
         constraints - Commands for handling constraints.
             migrate - Executes a database migration to use the latest Dolt data format.
         read-tables - Fetch table(s) at a specific commit into a new dolt repo
                  gc - Cleans up unreferenced data from the repository.
       filter-branch - Edits the commit history using the provided query.
          merge-base - Find the common ancestor of two commits.
             version - Displays the current Dolt cli version.
                dump - Export all tables in the working set into a file.

Installation

Dolt is a single ~103 megabyte program.

dolt $ du -h /Users/timsehn/go/bin/dolt
103M	/Users/timsehn/go/bin/dolt

It's really easy to install. Download it and put it on your PATH. We have a bunch of ways to make this even easier for most platforms.

From Latest Release

To install on Linux or Mac based systems run this command in your terminal:

sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash'

This will download the latest dolt release and put it in /usr/local/bin/, which is probably on your $PATH.

The install script needs sudo in order to put dolt in /usr/local/bin. If you don't have root privileges or aren't comfortable running a script with them, you can download the dolt binary for your platform from the latest release, unzip it, and put the binary somewhere on your $PATH.

Linux

Arch Linux

Dolt is packaged in the official repositories for Arch Linux.

pacman -S dolt

Mac

Homebrew

Dolt is on Homebrew, updated every release.

brew install dolt

MacPorts

On macOS, Dolt can also be installed via a community-managed port via MacPorts:

sudo port install dolt

Windows

Download the latest Microsoft Installer (.msi file) in releases and run it.

For information on running on Windows, see here.

Chocolatey

You can install dolt using Chocolatey:

choco install dolt

Docker

There are following official Docker images for Dolt:

From Source

Make sure you have Go installed, and that go is in your path. Dolt has a dependency on cgo, so you will need a working C compiler and toolchain as well.

Clone this repository and cd into the go directory. Then run:

go install ./cmd/dolt

The output will be in $GOPATH/bin, which defaults to ~/go/bin. To test your build, try:

~/go/bin/dolt version

Configuration

Verify that your installation has succeeded by running dolt in your terminal.

$ dolt
Valid commands for dolt are
[...]

Configure dolt with your user name and email, which you'll need to create commits. The commands work exactly the same as git.

$ dolt config --global --add user.email [email protected]
$ dolt config --global --add user.name "YOUR NAME"

Getting started

Navigate to the directory where you would like your data stored

Dolt needs a place to store your databases. I'm going to put my databases in ~/dolt.

% cd ~
% mkdir dolt
% cd dolt

Any databases you create will be stored in this directory. So, for this example, a directory named getting_started will be created here once you run create database getting_started. Navigating to ~/dolt/getting_started will allow you to access this database using the Dolt command line.

NOTE: For this example, the getting_started directory will be created after you run create database getting_started; in a SQL shell in the Create a schema section. Don't do anything except make the directory and navigate to it just yet.

Start a MySQL-compatible database server

Dolt ships with a MySQL compatible database server built in. To start it you use the command dolt sql-server. Running this command starts the server on port 3306.

dolt sql-server
Starting server with Config HP="localhost:3306"|T="28800000"|R="false"|L="info"

Your terminal will just hang there. This means the server is running. Any errors will be printed in this terminal. Just leave it there and open a new terminal.

Connect with any MySQL client

In the new terminal, we will now connect to the running database server using a client. Dolt also ships with a MySQL compatible client.

% dolt -u root -p "" sql
# Welcome to the Dolt MySQL client.
# Statements must be terminated with ';'.
# "exit" or "quit" (or Ctrl-D) to exit.
mysql>

In the other terminal where you ran dolt sql-server, you'll see the following log line.

2022-06-06T13:14:32-07:00 INFO [conn 1] NewConnection {DisableClientMultiStatements=false}

You are connected!

While we're here let's grab a copy of MySQL so we can connect with that client. Head over to the MySQL Getting Started documentation and install MySQL on your machine. I used Homebrew to install MySQL on my Mac.

MySQL comes with a MySQL server called mysqld and a MySQL client called mysql. You're only interested in the client. After following the instructions from MySQL's documentation, make sure you have a copy of the mysql client on your path:

% mysql --version
mysql  Ver 8.0.29 for macos12.2 on x86_64 (Homebrew)

Now, to connect the mysql client to Dolt, you are going to force the MySQL client through the TCP interface by passing in a host and port. The default is the socket interface which Dolt supports, but is only available on localhost. So, it's better to show off the TCP interface. The MySQL client also requires you specify a user, in this case root.

% mysql --host 127.0.0.1 --port 3306 -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.9-Vitess 

Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Again, to ensure the client actually connected, you should see the following in the dolt sql-server terminal

2022-06-06T13:26:55-07:00 INFO [conn 2] NewConnection {DisableClientMultiStatements=false}

As you can see, Dolt supports any MySQL-compatible client. Dolt ships with a client but you can use any MySQL client, like the one that comes with MySQL.

Create a schema

Now we're actually ready to do something interesting. I'll stay in the mysql client and execute the following SQL statements to create a database called getting_started. The getting_started database will have three tables: employees, teams, and employees_teams.

mysql> create database getting_started;
Query OK, 1 row affected (0.04 sec)

mysql> use getting_started;
Database changed
mysql> create table employees (
    id int, 
    last_name varchar(255), 
    first_name varchar(255), 
    primary key(id));
Query OK, 0 rows affected (0.01 sec)

mysql> create table teams (
    id int, 
    team_name varchar(255), 
    primary key(id)); 
Query OK, 0 rows affected (0.00 sec)

mysql> create table employees_teams(
    team_id int, 
    employee_id int, 
    primary key(team_id, employee_id), 
    foreign key (team_id) references teams(id), 
    foreign key (employee_id) references employees(id));
Query OK, 0 rows affected (0.01 sec)

mysql> show tables;
+---------------------------+
| Tables_in_getting_started |
+---------------------------+
| employees                 |
| employees_teams           |
| teams                     |
+---------------------------+
3 rows in set (0.00 sec)

Dolt supports foreign keys, secondary indexes, triggers, check constraints, and stored procedures. It's a modern, feature-rich SQL database.

Make a Dolt commit

It's time to use your first Dolt feature. We're going to make a Dolt commit. A Dolt commit allows you to time travel and see lineage. Make a Dolt commit whenever you want to restore or compare to this point in time.

Dolt exposes version control functionality through a Git-style interface. On the command line, Dolt commands map exactly to their Git equivalent with the targets being tables instead of files. In SQL, Dolt exposes version control read operations as system tables and version control write operations as stored procedures.

The naming of the system tables and stored procedures follows the dolt_<command> pattern. So dolt add on the CLI becomes dolt_add as a stored procedure. Passing options also follows the command line model. For instance, to specify tables to add, send the table names in as options to the dolt_add procedure. For named arguments like sending a message into the dolt_commit command use two arguments in sequence like ('-m', 'This is a message'). If you know Git, the version control procedures and system tables should feel familiar.

So, we add and commit our new schema like so.

mysql> call dolt_add('teams', 'employees', 'employees_teams');
+--------+
| status |
+--------+
|      0 |
+--------+
1 row in set (0.03 sec)

mysql> call dolt_commit('-m', 'Created initial schema');
+----------------------------------+
| hash                             |
+----------------------------------+
| ne182jemgrlm8jnjmoubfqsstlfi1s98 |
+----------------------------------+
1 row in set (0.02 sec)

mysql> select * from dolt_log;
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
| commit_hash                      | committer | email           | date                    | message                    |
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema     |
| vluuhvd0bn59598utedt77ed9q5okbcb | Tim Sehn  | [email protected] | 2022-06-07 16:33:59.531 | Initialize data repository |
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
2 rows in set (0.01 sec)

There you have it. Your schema is created and you have a Dolt commit tracking the creation, as seen in the dolt_log system table.

Note, a Dolt commit is different than a standard SQL transaction COMMIT. In this case, I am running the database with AUTOCOMMIT on, so each SQL statement is automatically generating a transaction COMMIT. If you want system to generate a Dolt commit for every transaction use the system variable, @@dolt_transaction_commit.

Insert some data

Now, I'm going to populate the database with a few employees here at DoltHub. Then, I'll assign the employees to two teams: engineering and sales. The CEO wears many hats at a start up so he'll be assigned to multiple teams.

mysql> insert into employees values 
    (0, 'Sehn', 'Tim'), 
    (1, 'Hendriks', 'Brian'), 
    (2, 'Son','Aaron'), 
    (3, 'Fitzgerald', 'Brian');
Query OK, 4 rows affected (0.01 sec)

mysql> select * from employees where first_name='Brian';
+------+------------+------------+
| id   | last_name  | first_name |
+------+------------+------------+
|    1 | Hendriks   | Brian      |
|    3 | Fitzgerald | Brian      |
+------+------------+------------+
2 rows in set (0.00 sec)

mysql> insert into teams values 
    (0, 'Engineering'), 
    (1, 'Sales');
Query OK, 2 rows affected (0.00 sec)

mysql> insert into employees_teams values 
    (0,0), 
    (1,0), 
    (2,0), 
    (0,1), 
    (3,1);
ERROR 1452 (HY000): cannot add or update a child row - Foreign key violation on fk: `rv9ek7ft`, table: `employees_teams`, referenced table: `teams`, key: `[2]`

Oops, I violated a constraint. It looks like I created the table with teams before employees. You should always specify your columns when you insert, not rely on natural ordering. Serves me right! Dolt comes with the full power of a modern SQL relational database to ensure data integrity.

mysql> insert into employees_teams(employee_id, team_id) values 
    (0,0), 
    (1,0), 
    (2,0), 
    (0,1), 
    (3,1);
Query OK, 5 rows affected (0.01 sec)

mysql> select first_name, last_name, team_name from employees 
    join employees_teams on (employees.id=employees_teams.employee_id) 
    join teams on (teams.id=employees_teams.team_id) 
    where team_name='Engineering';
+------------+-----------+-------------+
| first_name | last_name | team_name   |
+------------+-----------+-------------+
| Tim        | Sehn      | Engineering |
| Brian      | Hendriks  | Engineering |
| Aaron      | Son       | Engineering |
+------------+-----------+-------------+
3 rows in set (0.00 sec)

Looks like everything is inserted and correct. I was able to list the members of the engineering team using that three table JOIN. Dolt supports up to twelve table JOINs. Again, Dolt is a modern SQL relational database paired with Git-style version control.

Examine the diff

Now, what if you want to see what changed in your working set before you make a commit? You use the dolt_status and dolt_diff_<tablename> system tables.

mysql> select * from dolt_status;
+-----------------+--------+----------+
| table_name      | staged | status   |
+-----------------+--------+----------+
| teams           |      0 | modified |
| employees       |      0 | modified |
| employees_teams |      0 | modified |
+-----------------+--------+----------+
3 rows in set (0.01 sec)

mysql> select * from dolt_diff_employees;
+--------------+---------------+-------+-----------+----------------+----------------+-----------------+---------+----------------------------------+-------------------------+-----------+
| to_last_name | to_first_name | to_id | to_commit | to_commit_date | from_last_name | from_first_name | from_id | from_commit                      | from_commit_date        | diff_type |
+--------------+---------------+-------+-----------+----------------+----------------+-----------------+---------+----------------------------------+-------------------------+-----------+
| Sehn         | Tim           |     0 | WORKING   | NULL           | NULL           | NULL            |    NULL | ne182jemgrlm8jnjmoubfqsstlfi1s98 | 2022-06-07 16:35:49.277 | added     |
| Hendriks     | Brian         |     1 | WORKING   | NULL           | NULL           | NULL            |    NULL | ne182jemgrlm8jnjmoubfqsstlfi1s98 | 2022-06-07 16:35:49.277 | added     |
| Son          | Aaron         |     2 | WORKING   | NULL           | NULL           | NULL            |    NULL | ne182jemgrlm8jnjmoubfqsstlfi1s98 | 2022-06-07 16:35:49.277 | added     |
| Fitzgerald   | Brian         |     3 | WORKING   | NULL           | NULL           | NULL            |    NULL | ne182jemgrlm8jnjmoubfqsstlfi1s98 | 2022-06-07 16:35:49.277 | added     |
+--------------+---------------+-------+-----------+----------------+----------------+-----------------+---------+----------------------------------+-------------------------+-----------+
4 rows in set (0.00 sec)

As you can see from the diff I've added the correct values to the employees table. The values were previously NULL and now they are populated.

Let's finish off with another Dolt commit this time adding all effected tables using -am.

mysql> call dolt_commit('-am', 'Populated tables with data');
+----------------------------------+
| hash                             |
+----------------------------------+
| 13qfqa5rojq18j84d1n2htjkm6fletg4 |
+----------------------------------+
1 row in set (0.02 sec)

You can inspect the log using dolt_log and see which tables changed in each commit using an unscoped dolt_diff. Unscoped dolt_diff tells you whether schema, data, or both changed in that particular commit for the table.

mysql> select * from dolt_log;
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
| commit_hash                      | committer | email           | date                    | message                    |
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | Tim Sehn  | [email protected] | 2022-06-07 16:39:32.066 | Populated tables with data |
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema     |
| vluuhvd0bn59598utedt77ed9q5okbcb | Tim Sehn  | [email protected] | 2022-06-07 16:33:59.531 | Initialize data repository |
+----------------------------------+-----------+-----------------+-------------------------+----------------------------+
3 rows in set (0.00 sec)

mysql> select * from dolt_diff;
+----------------------------------+-----------------+-----------+-----------------+-------------------------+----------------------------+-------------+---------------+
| commit_hash                      | table_name      | committer | email           | date                    | message                    | data_change | schema_change |
+----------------------------------+-----------------+-----------+-----------------+-------------------------+----------------------------+-------------+---------------+
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | teams           | Tim Sehn  | [email protected] | 2022-06-07 16:39:32.066 | Populated tables with data |           1 |             0 |
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | employees       | Tim Sehn  | [email protected] | 2022-06-07 16:39:32.066 | Populated tables with data |           1 |             0 |
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | employees_teams | Tim Sehn  | [email protected] | 2022-06-07 16:39:32.066 | Populated tables with data |           1 |             0 |
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | employees       | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema     |           0 |             1 |
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | employees_teams | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema     |           0 |             1 |
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | teams           | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema     |           0 |             1 |
+----------------------------------+-----------------+-----------+-----------------+-------------------------+----------------------------+-------------+---------------+
6 rows in set (0.00 sec)

Oh no! I made a mistake.

Dolt supports undoing changes via call dolt_reset(). Let's imagine I accidentally drop a table.

mysql> drop table employees_teams;
Query OK, 0 rows affected (0.01 sec)

mysql> show tables;
+---------------------------+
| Tables_in_getting_started |
+---------------------------+
| employees                 |
| teams                     |
+---------------------------+
2 rows in set (0.00 sec)

In a traditional database, this could be disastrous. In Dolt, you're one command away from getting your table back.

mysql> call dolt_reset('--hard');
+--------+
| status |
+--------+
|      0 |
+--------+
1 row in set (0.01 sec)

mysql> show tables;
+---------------------------+
| Tables_in_getting_started |
+---------------------------+
| employees                 |
| employees_teams           |
| teams                     |
+---------------------------+
3 rows in set (0.01 sec)

Dolt makes operating databases less error prone. You can always back out changes you have in progress or rewind to a known good state. You also have the ability to undo specific commits using dolt_revert(). Even if you accidentally run drop database on the wrong database, Dolt lets you undo that by calling the dolt_undrop() stored procedure.

See the data in a SQL Workbench

Hate the command line? Let's use Tableplus to make some modifications. Tableplus is a free SQL Workbench. Follow the installation instructions from their website.

Now, to connect you must select MySQL as the connection type. Then enter a name for your connection, getting_started as your database, and root as your user.

Tableplus Connection

Click connect and you'll be presented with a familiar database workbench GUI.

Tableplus

Make changes on a branch

To make changes on a branch, I use the dolt_checkout() stored procedure. Using the -b option creates a branch, just like in Git.

Tableplus gives me the ability to enter a multiple line SQL script on the SQL tab. I entered the following SQL to checkout a branch, update, insert, delete, and finally Dolt commit my changes.

call dolt_checkout('-b','modifications');
update employees SET first_name='Timothy' where first_name='Tim';
insert INTO employees (id, first_name, last_name) values (4,'Daylon', 'Wilkins');
insert into employees_teams(team_id, employee_id) values (0,4);
delete from employees_teams where employee_id=0 and team_id=1;
call dolt_commit('-am', 'Modifications on a branch');

Here's the result in Tableplus.

New Updates

Back in my terminal, I cannot see the table modifications made in Tableplus because they happened on a different branch than the one I have checked out in my session.

mysql> select * from dolt_branches;
+---------------+----------------------------------+------------------+------------------------+-------------------------+----------------------------+
| name          | hash                             | latest_committer | latest_committer_email | latest_commit_date      | latest_commit_message      |
+---------------+----------------------------------+------------------+------------------------+-------------------------+----------------------------+
| main          | 13qfqa5rojq18j84d1n2htjkm6fletg4 | Tim Sehn         | [email protected]        | 2022-06-07 16:39:32.066 | Populated tables with data |
| modifications | uhkv57j4bp2v16vcnmev9lshgkqq8ppb | Tim Sehn         | [email protected]        | 2022-06-07 16:41:49.847 | Modifications on a branch  |
+---------------+----------------------------------+------------------+------------------------+-------------------------+----------------------------+
2 rows in set (0.00 sec)

mysql> select active_branch();
+-----------------+
| active_branch() |
+-----------------+
| main            |
+-----------------+
1 row in set (0.00 sec)

mysql> select * from employees;
+------+------------+------------+
| id   | last_name  | first_name |
+------+------------+------------+
|    0 | Sehn       | Tim        |
|    1 | Hendriks   | Brian      |
|    2 | Son        | Aaron      |
|    3 | Fitzgerald | Brian      |
+------+------------+------------+
4 rows in set (0.00 sec)

I can query the branch no matter what I have checked out using SQL as of syntax.

mysql> select * from employees as of 'modifications';
+------+------------+------------+
| id   | last_name  | first_name |
+------+------------+------------+
|    0 | Sehn       | Timothy    |
|    1 | Hendriks   | Brian      |
|    2 | Son        | Aaron      |
|    3 | Fitzgerald | Brian      |
|    4 | Wilkins    | Daylon     |
+------+------------+------------+
5 rows in set (0.01 sec)

If I'd like to see the diff between the two branches, I can use the dolt_diff() table function. It takes two branches and the table name as arguments.

mysql> select * from dolt_diff('main', 'modifications', 'employees');
+--------------+---------------+-------+---------------+-------------------------+----------------+-----------------+---------+-------------+-------------------------+-----------+
| to_last_name | to_first_name | to_id | to_commit     | to_commit_date          | from_last_name | from_first_name | from_id | from_commit | from_commit_date        | diff_type |
+--------------+---------------+-------+---------------+-------------------------+----------------+-----------------+---------+-------------+-------------------------+-----------+
| Sehn         | Timothy       |     0 | modifications | 2022-06-07 16:41:49.847 | Sehn           | Tim             |       0 | main        | 2022-06-07 16:39:32.066 | modified  |
| Wilkins      | Daylon        |     4 | modifications | 2022-06-07 16:41:49.847 | NULL           | NULL            |    NULL | main        | 2022-06-07 16:39:32.066 | added     |
+--------------+---------------+-------+---------------+-------------------------+----------------+-----------------+---------+-------------+-------------------------+-----------+
2 rows in set (0.00 sec)

As you can see, you have the full power of Git-style branches and diffs in a SQL database with Dolt.

Make a schema change on another branch

I can also make schema changes on branches for isolated testing of new schema. I'm going to add a start_date column on a new branch and populate it.

mysql> call dolt_checkout('-b', 'schema_changes');
+--------+
| status |
+--------+
|      0 |
+--------+
1 row in set (0.01 sec)

mysql> alter table employees add column start_date date;
Query OK, 0 rows affected (0.02 sec)

mysql> update employees set start_date='2018-09-08';
Query OK, 4 rows affected (0.01 sec)
Rows matched: 4  Changed: 4  Warnings: 0

mysql> update employees set start_date='2021-04-19' where last_name='Fitzgerald';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select * from employees;
+------+------------+------------+------------+
| id   | last_name  | first_name | start_date |
+------+------------+------------+------------+
|    0 | Sehn       | Tim        | 2018-09-08 |
|    1 | Hendriks   | Brian      | 2018-09-08 |
|    2 | Son        | Aaron      | 2018-09-08 |
|    3 | Fitzgerald | Brian      | 2021-04-19 |
+------+------------+------------+------------+
4 rows in set (0.00 sec)

mysql> call dolt_commit('-am', 'Added start_date column to employees');
+----------------------------------+
| hash                             |
+----------------------------------+
| pg3nfi0j1dpc5pf1rfgckpmlteaufdrt |
+----------------------------------+
1 row in set (0.01 sec)

Changing schema on a branch gives you a new method for doing isolated integration testing of new schema changes.

Merge it all together

Let's assume all the testing of the new schema on the schema_changes branch and data on the modifications branch completed flawlessly. It's time to merge all our edits together onto main. This is done using the dolt_merge stored procedure.

mysql> call dolt_checkout('main');
+--------+
| status |
+--------+
|      0 |
+--------+
1 row in set (0.01 sec)

mysql> select * from dolt_status;
Empty set (0.00 sec)

mysql> call dolt_merge('schema_changes');
+--------------+
| no_conflicts |
+--------------+
|            1 |
+--------------+
1 row in set (0.01 sec)

mysql> select * from employees;
+------+------------+------------+------------+
| id   | last_name  | first_name | start_date |
+------+------------+------------+------------+
|    0 | Sehn       | Tim        | 2018-09-08 |
|    1 | Hendriks   | Brian      | 2018-09-08 |
|    2 | Son        | Aaron      | 2018-09-08 |
|    3 | Fitzgerald | Brian      | 2021-04-19 |
+------+------------+------------+------------+
4 rows in set (0.00 sec)

Schema change successful. We now have start dates. Data changes are next.

mysql> call dolt_merge('modifications');
+--------------+
| no_conflicts |
+--------------+
|            1 |
+--------------+
1 row in set (0.02 sec)

mysql> select * from employees;
+------+------------+------------+------------+
| id   | last_name  | first_name | start_date |
+------+------------+------------+------------+
|    0 | Sehn       | Timothy    | 2018-09-08 |
|    1 | Hendriks   | Brian      | 2018-09-08 |
|    2 | Son        | Aaron      | 2018-09-08 |
|    3 | Fitzgerald | Brian      | 2021-04-19 |
|    4 | Wilkins    | Daylon     | NULL       |
+------+------------+------------+------------+
5 rows in set (0.00 sec)

Data changes successful as well. As you can see, I am now "Timothy" instead of "Tim", Daylon is added, and we all have start dates except for Daylon who was added on a different branch.

mysql> select first_name, last_name, team_name from employees 
    join employees_teams on (employees.id=employees_teams.employee_id) 
    join teams on (teams.id=employees_teams.team_id) 
    where team_name='Sales';
+------------+------------+-----------+
| first_name | last_name  | team_name |
+------------+------------+-----------+
| Brian      | Fitzgerald | Sales     |
+------------+------------+-----------+
1 row in set (0.01 sec)

I'm also gone from the Sales Team. Engineering is life.

I have to commit all my changes because the last merge was not a fast-forward merge.

mysql> call dolt_commit('-m', 'Merged all branches');
+----------------------------------+
| hash                             |
+----------------------------------+
| vn9b0qcematsj2f6ka0hfoflhr5s6p0b |
+----------------------------------+
1 row in set (0.01 sec)

mysql> select * from dolt_log;
+----------------------------------+-----------+-----------------+-------------------------+--------------------------------------+
| commit_hash                      | committer | email           | date                    | message                              |
+----------------------------------+-----------+-----------------+-------------------------+--------------------------------------+
| vn9b0qcematsj2f6ka0hfoflhr5s6p0b | Tim Sehn  | [email protected] | 2022-06-07 17:10:02.07  | Merged all branches                  |
| pg3nfi0j1dpc5pf1rfgckpmlteaufdrt | Tim Sehn  | [email protected] | 2022-06-07 16:44:37.513 | Added start_date column to employees |
| uhkv57j4bp2v16vcnmev9lshgkqq8ppb | Tim Sehn  | [email protected] | 2022-06-07 16:41:49.847 | Modifications on a branch            |
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | Tim Sehn  | [email protected] | 2022-06-07 16:39:32.066 | Populated tables with data           |
| ne182jemgrlm8jnjmoubfqsstlfi1s98 | Tim Sehn  | [email protected] | 2022-06-07 16:35:49.277 | Created initial schema               |
| vluuhvd0bn59598utedt77ed9q5okbcb | Tim Sehn  | [email protected] | 2022-06-07 16:33:59.531 | Initialize data repository           |
+----------------------------------+-----------+-----------------+-------------------------+--------------------------------------+
6 rows in set (0.00 sec)

Now, we have a database with all the schema and data changes merged and ready for use.

Audit Cell Lineage

Which commit changed my first name? With Dolt you have lineage for every cell in your database. Let's use the dolt_history_<tablename> and dolt_diff_<tablename> to explore the lineage features in Dolt.

dolt_history_<tablename> shows you the state of the row at every commit.

mysql> select * from dolt_history_employees where id=0 order by commit_date;
+------+-----------+------------+------------+----------------------------------+-----------+-------------------------+
| id   | last_name | first_name | start_date | commit_hash                      | committer | commit_date             |
+------+-----------+------------+------------+----------------------------------+-----------+-------------------------+
|    0 | Sehn      | Tim        | NULL       | 13qfqa5rojq18j84d1n2htjkm6fletg4 | Tim Sehn  | 2022-06-07 16:39:32.066 |
|    0 | Sehn      | Timothy    | NULL       | uhkv57j4bp2v16vcnmev9lshgkqq8ppb | Tim Sehn  | 2022-06-07 16:41:49.847 |
|    0 | Sehn      | Tim        | 2018-09-08 | pg3nfi0j1dpc5pf1rfgckpmlteaufdrt | Tim Sehn  | 2022-06-07 16:44:37.513 |
|    0 | Sehn      | Timothy    | 2018-09-08 | vn9b0qcematsj2f6ka0hfoflhr5s6p0b | Tim Sehn  | 2022-06-07 17:10:02.07  |
+------+-----------+------------+------------+----------------------------------+-----------+-------------------------+
4 rows in set (0.00 sec)

dolt_diff_<tablename> allows you to filter the history down to only commits when the cell in question changed. In this case, I'm interested in the commits that are changing my first name. Note, there are two commits that changed my name because one is the original change and the second is the merge commit.

mysql> select to_commit,from_first_name,to_first_name from dolt_diff_employees 
    where (from_id=0 or to_id=0) and (from_first_name <> to_first_name or from_first_name is NULL)
    order by to_commit_date;
+----------------------------------+-----------------+---------------+
| to_commit                        | from_first_name | to_first_name |
+----------------------------------+-----------------+---------------+
| 13qfqa5rojq18j84d1n2htjkm6fletg4 | NULL            | Tim           |
| uhkv57j4bp2v16vcnmev9lshgkqq8ppb | Tim             | Timothy       |
| vn9b0qcematsj2f6ka0hfoflhr5s6p0b | Tim             | Timothy       |
+----------------------------------+-----------------+---------------+
3 rows in set (0.01 sec)

Dolt provides powerful data audit capabilities down to individual cells. When, how, and why has each cell in your database changed over time?

Additional Reading

Head over to our documentation now that you have a feel for Dolt. You can also read about what we've been working on in our blog.

Security Policy

Dolt's current security policy is maintained in this repository. Please follow the disclosure instructions there. Please do not initially report security issues in this repository's public GitHub issues.

Credits and License

Dolt relies heavily on open source code and ideas from the Noms project. We are very thankful to the Noms team for making this code freely available, without which we would not have been able to build Dolt so rapidly.

Dolt is licensed under the Apache License, Version 2.0. See LICENSE for details.

dolt's People

Contributors

aboodman avatar alrs avatar andrew-wm-arthur avatar andy-wm-arthur avatar arv avatar bheni avatar cmasone-attic avatar coffeegoddd avatar dependabot-preview[bot] avatar druvv avatar fulghum avatar hydrocharged avatar jennifersp avatar jycor avatar kalman avatar liuliu-dev avatar macneale4 avatar max-hoffman avatar mikegray avatar nicktobey avatar oscarbatori avatar pavelsafronov avatar reltuk avatar solipsis-project avatar stephkyou avatar tbantle22 avatar timsehn avatar vinairachakonda avatar willhite avatar zachmu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dolt's Issues

Dolt: Bad error on excel import of non excel file

timsehn:bats-test timsehn$ dolt table import -c employees ~/liquidata/git/ld/dolt/bats/helper/bad.xlsx
Error creating reader for /Users/timsehn/liquidata/git/ld/dolt/bats/helper/bad.xlsx.
When attempting to move data from xlsx file:/Users/timsehn/liquidata/git/ld/dolt/bats/helper/bad.xlsx to dolt table:employees, could not open a reader.
cause: zip: not a valid zip file

Should say "not a valid xlsx file".

Dolt: Can create a table with a blob type. Not supposed to be able to.

timsehn:bats-test timsehn$ cat 1pkalltypes.schema 
{
  "columns": [
    {
      "tag": 0,
      "name": "pk",
      "kind": "int",
      "is_part_of_pk": true,
      "col_constraints": [
        {
          "constraint_type": "not_null",
          "params": null
        }
      ]
    },
    {
      "tag": 1,
      "name": "int",
      "kind": "int",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 2,
      "name": "string",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 3,
      "name": "boolean",
      "kind": "boolean",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 4,
      "name": "float",
      "kind": "float",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 5,
      "name": "uint",
      "kind": "uint",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {	
      "tag": 6,
      "name": "uuid",
      "kind": "uuid",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 7,
      "name": "blob",
      "kind": "blob",
      "is_part_of_pk": false,
      "col_constraints": []
    }
  ]
}timsehn:bats-test timsehn$ dolt init
Successfully initialized dolt data repository.
timsehn:bats-test timsehn$ dolt table create -s=1pkalltypes.schema test
timsehn:bats-test timsehn$ dolt table schema
test @ working
{
  "columns": [
    {
      "tag": 0,
      "name": "pk",
      "kind": "int",
      "is_part_of_pk": true,
      "col_constraints": [
        {
          "constraint_type": "not_null",
          "params": null
        }
      ]
    },
    {
      "tag": 1,
      "name": "int",
      "kind": "int",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 2,
      "name": "string",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 3,
      "name": "boolean",
      "kind": "bool",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 4,
      "name": "float",
      "kind": "float",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 5,
      "name": "uint",
      "kind": "uint",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 6,
      "name": "uuid",
      "kind": "uuid",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 7,
      "name": "blob",
      "kind": "blob",
      "is_part_of_pk": false,
      "col_constraints": []
    }
  ]
}

timsehn:bats-test timsehn$ 

Store default column values in schema

We only handle default column values when a column is added. To be consistent with other SQL databases, we need to store any default values are part of the schema and set those values on an insert as appropriate.

Dolt: Bad error message for invalid schema on json import

timsehn:bats-test timsehn$ dolt table import -c -s ~/liquidata/git/ld/dolt/bats/helper/employees-sch-bad.json employees ~/liquidata/git/ld/dolt/bats/helper/employees-tbl.json 
Error creating reader for /Users/timsehn/liquidata/git/ld/dolt/bats/helper/employees-tbl.json.
When attempting to move data from json file:/Users/timsehn/liquidata/git/ld/dolt/bats/helper/employees-tbl.json to dolt table:employees, could not open a reader.
cause: invalid character 'b' after object key:value pair

Mentions the valid file not the invalid file.

dolt: push and pull a remote repository should say "Already up to date"

timsehn:bats-test timsehn$ remotesrv --dir /var/tmp/ &>/dev/null &
[1] 17282
timsehn:bats-test timsehn$ dolt pull test-remote
Updating nf4l8aabdndbmstk938u978epq2fjbjv..nf4l8aabdndbmstk938u978epq2fjbjv
Fast-forward

Notice the same commit numbers. Should say "Already up to date." in this case.

Failing bats test created.

Dolt merge murdered my table?

Made schema change on a branch. Kept same data. Merged that branch to master. Committed. Table seems gone as in dolt table cat employees returns empty. dolt table schema shows the old one still there.

timsehn:dolt-video timsehn$ dolt checkout change-schema 
Switched to branch 'change-schema'
timsehn:dolt-video timsehn$ dolt table cat employees
id | first name | last name | title            
0  | tim        | sehn      | ceo              
1  | aaron      | son       | founder          
2  | brian      | hendriks  | founder          
3  | matt       | jesuele   | software engineer
timsehn:dolt-video timsehn$ dolt checkout master
Switched to branch 'master'
timsehn:dolt-video timsehn$ dolt log
commit uuhe2fgkg9pje89d74aetosg3jadjj0p
Merge: qccmdrkre0ld2mf63cpnmik5lcr32pjb c008futlhdr2du3un49a8mkvqrbi240q
Author: tim sehn <[email protected]>
Date:   Tue Feb 05 15:57:49 -0800 2019

	Merged new schema. Changed id to int and required

commit c008futlhdr2du3un49a8mkvqrbi240q
Merge: qccmdrkre0ld2mf63cpnmik5lcr32pjb ilhik5deso56fa1p67t7ugo5oimsnhje
Author: tim sehn <[email protected]>
Date:   Tue Feb 05 15:42:05 -0800 2019

	Changed schema. id from string to int and required.

commit ilhik5deso56fa1p67t7ugo5oimsnhje
Author: tim sehn <[email protected]>
Date:   Fri Feb 01 12:40:26 -0800 2019

	add start date

commit qccmdrkre0ld2mf63cpnmik5lcr32pjb
Author: tim sehn <[email protected]>
Date:   Thu Jan 31 15:51:01 -0800 2019

	Added Matt to employees

commit kldk7psjq4fttdujg65mthevpmduhd59
Author: tim sehn <[email protected]>
Date:   Thu Jan 31 15:42:24 -0800 2019

	Initial employees table

commit f7lhvs2of183hrbo1b2n81htjpifchii
Author: tim sehn <[email protected]>
Date:   Thu Jan 31 15:33:12 -0800 2019

	Data repository created.

timsehn:dolt-video timsehn$ dolt table cat employees
timsehn:dolt-video timsehn$ dolt table schema
employees @ working
idx | name       | type   | nullable | primary key
0   | id         | string | true     | true       
1   | first name | string | true     | false      
2   | last name  | string | true     | false      
3   | title      | string | true     | false  

--ours nor --theirs useful in conflict resolution

timsehn:test timsehn$ dolt conflicts cat employees
 +  | ours   | 3 | matt | jesuele | software engineer | 12/3/2018 |        
 +  | theirs | 3 | matt | jesuele | software engineer |           | current

I really want to specify which fields to keep from our and theirs. Maybe a third option like --choose. --pk:[value] field1:ours field2:theirs....

Dolt: Diff with two revisions is backwards

In git, if you do:

git diff older_commit newer_commit

it shows the diff with the older commit as the base. Dolt currently does the opposite. This is probably not what we want:

$ dolt diff lh7tipknpuolrahdasvflvral4n72s32 r52ivkv7jkje3pbaedcg0f6idnb93ek6
diff --dolt a/submissions b/submissions
--- a/submissions @ 2vtm8bdmg9k7k2sff8907etf596p639j
+++ b/submissions @ sjb1tlf6ot137jfn84a6eaujn7ufmh5e
+-----+--------------------------------------+--------------------------+-----------------+---------+-------------+----------------------+
|     | id                                   | email                    | name            | company | hear_source | created_at           |
+-----+--------------------------------------+--------------------------+-----------------+---------+-------------+----------------------+
|  -  | 1f8ef759-367d-4b2c-bd91-85788fa2732c | [email protected] | Alessio Fanelli | <NULL>  | <NULL>      | 2019-05-27T00:30:37Z |
+-----+--------------------------------------+--------------------------+-----------------+---------+-------------+----------------------+

Dolt: dolt table select --hide-conflicts does not work

timsehn:bats-test timsehn$ dolt init
Successfully initialized dolt data repository.
timsehn:bats-test timsehn$ dolt table create -s 1pk5col.schema test
timsehn:bats-test timsehn$ dolt add test
timsehn:bats-test timsehn$ dolt commit -m "added test table"
timsehn:bats-test timsehn$ dolt branch test-branch
timsehn:bats-test timsehn$ dolt table put-row test pk:0 c1:1 c2:2 c3:3 c4:4 c5:5
Successfully put row.
timsehn:bats-test timsehn$ dolt add test
timsehn:bats-test timsehn$ dolt commit -m "added test row"
timsehn:bats-test timsehn$ dolt checkout test-branch
Switched to branch 'test-branch'
timsehn:bats-test timsehn$ dolt table put-row test pk:0 c1:1 c2:2 c3:3 c4:4 c5:6
Successfully put row.
timsehn:bats-test timsehn$ dolt add test
timsehn:bats-test timsehn$ dolt commit -m "added conflicting test row"
timsehn:bats-test timsehn$ dolt checkout master
Switched to branch 'master'
timsehn:bats-test timsehn$ dolt merge test-branch
Updating 5u5pr8vuge4gmskldopa5e7kr0gftv2r..nlj137rb7et8bbcintgrqjjefhladou5
Auto-merging test
CONFLICT (content): Merge conflict in test
Automatic merge failed; fix conflicts and then commit the result.
timsehn:bats-test timsehn$ dolt table select test
Cnf|pk|c1|c2|c3|c4|c5
 ! |0 |1 |2 |3 |4 |5 
timsehn:bats-test timsehn$ dolt table select --hide-conflicts test
Cnf|pk|c1|c2|c3|c4|c5
 ! |0 |1 |2 |3 |4 |5 

There's a bats test for this issue.

dolt: got a panic when doing a merge from a remote with conflicts

timsehn:test-repo timsehn$ dolt pull origin
Updating bl0uhig0sr1ma64sltebj20blgv7gg76..16b1487q14e8d1fp68m081336o70m0s3
clone failed
cause: runtime error: invalid memory address or nil pointer dereference
timsehn:test-repo timsehn$ dolt ls
Tables in working set:
	 test2
timsehn:test-repo timsehn$ echo $?
0
timsehn:test-repo timsehn$ dolt fetch origin master
timsehn:test-repo timsehn$ dolt merge remotes/origin/master
Updating bl0uhig0sr1ma64sltebj20blgv7gg76..16b1487q14e8d1fp68m081336o70m0s3
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x182aa40]

goroutine 1 [running]:
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/doltdb.(*Table).HashOf(0x0, 0x0, 0x0, 0x0)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/doltdb/table.go:184 +0x40
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/merge.(*Merger).MergeTable(0xc000370b10, 0xc000334fc8, 0x5, 0xc00032efe0, 0x2, 0x2, 0x0)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/merge/merge.go:53 +0x1c5
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/env/actions.MergeCommits(0xc00042c000, 0xc0004085f0, 0xc000408730, 0xaf, 0x0, 0x0, 0x0)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/env/actions/merge.go:22 +0x1a9
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/commands.executeMerge(0xc0004160c0, 0xc0004085f0, 0xc000408730, 0x7ffeefbffb73, 0x15, 0xc0003f83a0, 0x20)
	/Users/timsehn/liquidata/git/ld/dolt/go/cmd/dolt/commands/merge.go:181 +0x5e
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/commands.mergeBranch(0xc0004160c0, 0x7ffeefbffb73, 0x15, 0x0, 0x25ce780)
	/Users/timsehn/liquidata/git/ld/dolt/go/cmd/dolt/commands/merge.go:141 +0x2fd
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/commands.Merge(0xc000422046, 0xa, 0xc0000b60b0, 0x1, 0x1, 0xc0004160c0, 0x5)
	/Users/timsehn/liquidata/git/ld/dolt/go/cmd/dolt/commands/merge.go:99 +0x3ae
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/cli.GenSubCommandHandler.func1(0x1c25659, 0x4, 0xc0000b60a0, 0x2, 0x2, 0xc0004160c0, 0xc000370401)
	/Users/timsehn/liquidata/git/ld/dolt/go/cmd/dolt/cli/command.go:61 +0x19b
main.main()
	/Users/timsehn/liquidata/git/ld/dolt/go/cmd/dolt/dolt.go:76 +0x256

There is a skipped bats test for this.

dolt: dolt config --list does not respect local overrides

timsehn:bats-test timsehn$ dolt config --global --add test global
Config successfully updated.
timsehn:bats-test timsehn$ dolt config --local --add test local
Config successfully updated.
timsehn:bats-test timsehn$ dolt config --list
test = local
testlocal = testlocal
test = global
user.email = [email protected]
user.name = bats tester
timsehn:bats-test timsehn$ dolt config --local --get test
local
timsehn:bats-test timsehn

There is a skipped bats test for this.

Dolt: Allow partial commit hashes in commands

Anywhere Git expects commit hashes, it allows you to provide just the beginning of the hash and it will attempt to match it, throwing an error if the partial hash is not enough to uniquely identify a commit.

It would be super helpful if Dolt did the same.

Dolt: Duplicate column name in put-row puts last. Should error?

timsehn:employees timsehn$ dolt table put-row employees id:4 "first name":faraj "last name":khasib title:"software engineer" "start date":12/3/2018 "last name":"12/5/2018" level:4 email:[email protected] nationality:palestinian
Successfully put row.
timsehn:employees timsehn$ dolt diff
diff --dolt a/employees b/employees
--- a/employees @ nkhkojanq4mlddpe7uh029bvtanfbfgj
+++ b/employees @ ocb61vb8n4g8fuofcl4rhiskh29kq3q4
   id | first name | last name | title             | start date | end date  | level | email              | nationality
 < 4  | faraj      | khasib    | software engineer | 12/3/2018  | 12/5/2018 | 4     | [email protected] | american   
 > 4  | faraj      | 12/5/2018 | software engineer | 12/3/2018  |           | 4     | [email protected] | palestinian

Doing CDC with Dolt

Exploring uses cases with Dolt.

Maybe the hub already does a change to the data is just like CDC, and git Servers tell you when someone makea commit. But with a CDC you want to be able to query and get an event when the data that is part of the Query Result Set changes.

PS. Polling is cheating !

Dolt: Improve error message for bad diff arguments

Git:

$ git diff badrevision
fatal: ambiguous argument 'badrevision': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

Dolt:

$ dolt diff badrevision
error: Unknown table: 'badrevision'

Dolt: Weird error message on Dolt diff when changing schema

timsehn:test1 timsehn$ dolt table import -c -f -s employees.schema employees employees.csv 
timsehn:test1 timsehn$ dolt diff
diff --dolt a/employees b/employees
--- a/employees @ rr223frevrus2cefsmrld6582924bq9o
+++ b/employees @ jsrqk409vahr5h8q69nevlle06114qs8
Failed to merge schemas
timsehn:test1 timsehn$ dolt table cat employees
id|first name|last name|title            |start date|end date
0 |tim       |sehn     |ceo              |8/6/2018  |current 
1 |aaron     |son      |founder          |8/6/2018  |current 
2 |brian     |hendriks |founder          |8/6/2018  |current 
3 |matt      |jesuele  |software engineer|12/3/2018 |current 
timsehn:test1 timsehn$ dolt table schema employees 
employees @ working
{
  "columns": [
    {
      "tag": 0,
      "name": "id",
      "kind": "int",
      "is_part_of_pk": true,
      "col_constraints": [
        {
          "constraint_type": "not_null",
          "params": null
        }
      ]
    },
    {
      "tag": 1,
      "name": "first name",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 2,
      "name": "last name",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 3,
      "name": "title",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 4,
      "name": "start date",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    },
    {
      "tag": 5,
      "name": "end date",
      "kind": "string",
      "is_part_of_pk": false,
      "col_constraints": []
    }
  ]
}

Basically export the schema and data for a table. Make a change to the type in the schema file (in this case string to int). Import the table schema and data using -c -f. Get a weird "Failed to merge schemas" on dolt diff. But when I output the schema and the table, things look fine.

Dolt: Panic on .psv import

timsehn:dictionary-english timsehn$ dolt table import -c -f -pk word,text raw ../../dolt-test-inputs/english-dictionary/parsed-dictionary.psv 
panic: runtime error: slice bounds out of range

goroutine 37 [running]:
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/untyped/csv.appendToken(0xc009c34000, 0x4, 0x4, 0xc009c27a80, 0x19ae, 0x7fd, 0x7fe, 0x0, 0xc009c34000, 0x4, ...)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/untyped/csv/line.go:44 +0x59a
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/untyped/csv.csvSplitLine(0xc009c27a80, 0x19ae, 0x10000007c, 0x0, 0x7493be8, 0x1)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/untyped/csv/line.go:11 +0xf1
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/untyped/csv.(*CSVReader).parseRow(0xc0003b8300, 0xc009c27a80, 0x19ae, 0x19ae, 0x0, 0x0, 0x0)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/untyped/csv/reader.go:128 +0x6a
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/untyped/csv.(*CSVReader).ReadRow(0xc0003b8300, 0x1dee040, 0xc0000bc010, 0xc00044e060, 0x0, 0x1894400, 0x0)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/untyped/csv/reader.go:101 +0x1e0
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/pipeline.ProcFuncForReader.func1(0x0, 0xc00044e060, 0xc00044e000, 0xc0003c4600, 0xc0003b2500)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/pipeline/procfunc_help.go:62 +0x41
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/pipeline.ProcFuncForSourceFunc.func1(0xc00044a080, 0xc0003c4660, 0xc0003c4600, 0xc00044e060)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/pipeline/procfunc_help.go:30 +0xa0
github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/pipeline.(*Pipeline).Start.func3(0xc00044a080, 0xc0003c4660)
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/pipeline/pipeline.go:214 +0x78
created by github.com/liquidata-inc/ld/dolt/go/libraries/doltcore/table/pipeline.(*Pipeline).Start
	/Users/timsehn/liquidata/git/ld/dolt/go/libraries/doltcore/table/pipeline/pipeline.go:212 +0x3b8

I uploaded the offending .psv to:

https://drive.google.com/open?id=1lJXuDSOQ0XRBQTXRbQHC5TwjFlXzPHbK

I'll try and get a reducible bats test.

dolt: dolt push when behind your remote returns 0 exit code. should be 1

timsehn:test-repo timsehn$ dolt push origin master
To localhost:50051/test-org/test-repo
! [rejected]          master -> remotes/origin/master (non-fast-forward)
error: failed to push some refs to 'localhost:50051/test-org/test-repo'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'dolt pull ...') before pushing again.
timsehn:test-repo timsehn$ echo $?
0

There is a failed bats test for this.

Dolt: `SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP())` Fails

When implementing dolt serve functionality, it was observed that Oracle's .NET MySQL connector would call SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP()) immediately when connecting to dolt. This is not currently a supported SQL function, and thus an exception was thrown by the .NET client.

Specifically, the exception states that utc_timestamp was not found.

Dolt: Dolt table export does not quote lists with commas

timsehn:lunch-places timsehn$ dolt table select lunch-places
name     |type of food|visitors                     |rating
Kazu Nori|sushi       |aaron,brian,katie,osheiza,tim|8.8   
timsehn:lunch-places timsehn$ dolt table export lunch-places brian.csv
Successfully exported data.
timsehn:lunch-places timsehn$ cat brian.csv 
name,type of food,visitors,rating
Kazu Nori,sushi,aaron,brian,katie,osheiza,tim,8.8

'dolt login' always creates new credentials, even if they are unncesssary.

It's probably rare that the user actually wants to create new credentials and associate them with a different dolthub account. If credentials already exist and they already work with the registered remote, should we prompt for behavior, or have the user pass --new-credentails or something if they really want to re-login?

Dolt: Says failed to merge schemas when changing strings to ints/floats

timsehn:bitcoin-coinbase timsehn$ dolt diff
diff --dolt a/minute_summary b/minute_summary
--- a/minute_summary @ ms4rpm098j6afj454e5uginlq6bslie0
+++ b/minute_summary @ itpfnfv7bkip2e1qt6k612v3b4p4l85f
  CREATE TABLE minute_summary (
<   Timestamp varchar not null comment 'tag:0'
>   Timestamp     int not null comment 'tag:0'
<   Open varchar comment 'tag:1'
>   Open   float comment 'tag:1'
<   High varchar comment 'tag:2'
>   High   float comment 'tag:2'
<   Low varchar comment 'tag:3'
>   Low   float comment 'tag:3'
<   Close varchar comment 'tag:4'
>   Close   float comment 'tag:4'
<   Volume_(BTC) varchar comment 'tag:5'
>   Volume_(BTC)   float comment 'tag:5'
<   Volume_(Currency) varchar comment 'tag:6'
>   Volume_(Currency)   float comment 'tag:6'
<   Weighted_Price varchar comment 'tag:7'
>   Weighted_Price   float comment 'tag:7'
  );

Failed to merge schemas

Import a table from a csv. Get all string columns. Export the schema to a file. Modify the file to have the types you want. Reimport but this time specify -s with the modified schema file.

I would expect this to show no data diff, not failed to merge schemas.

New file generated each time dolt table import -u is run

I attempted to update a table with a .csv file that contained the same data as the existing table, but with some of the rows missing. No updates were performed since no rows in this file were modified. However, each time this dolt table import -u is run, a new file of size 260bytes is created.

I'm not sure if this is intended behavior for a file to be created each time this command is run since no updates are made the first time this command was run, or on subsequent tries.

Dolt: Funky error message for duplicate row insert

timsehn:bats-test timsehn$ dolt sql -q "insert into test (pk,c1,c2,c3,c4,c5) values (0,6,6,6,6,6)"
Error inserting rows: [cannot insert existing row {map[0:0] map[5:6 1:6 2:6 3:6 4:6]}]

That map message is weird.

Dolt: Name a table and branch the same, dolt checkout ambiguous

Successfully initialized dolt data repository.
timsehn:bats-test timsehn$ dolt table create -s=1pk5col.schema test
timsehn:bats-test timsehn$ dolt add test
timsehn:bats-test timsehn$ dolt commit -m "create table"
timsehn:bats-test timsehn$ dolt table put-row test pk:0 c1:1 c2:2 c3:3 c4:4 c5:5
Successfully put row.
timsehn:bats-test timsehn$ dolt diff
diff --dolt a/test b/test
--- a/test @ nm261ggrdkpaitnt7g98ac0ktse89sl5
+++ b/test @ uu8j16uneeb7u0vk9ki4faa6tl53t2e8
   pk | c1 | c2 | c3 | c4 | c5
 + 0  | 1  | 2  | 3  | 4  | 5 
timsehn:bats-test timsehn$ dolt diff
diff --dolt a/test b/test
--- a/test @ nm261ggrdkpaitnt7g98ac0ktse89sl5
+++ b/test @ uu8j16uneeb7u0vk9ki4faa6tl53t2e8
   pk | c1 | c2 | c3 | c4 | c5
 + 0  | 1  | 2  | 3  | 4  | 5 
timsehn:bats-test timsehn$ dolt branch test
timsehn:bats-test timsehn$ dolt branch
* master                          
  test
timsehn:bats-test timsehn$ dolt checkout test
timsehn:bats-test timsehn$ dolt branch
* master                          
  test    

Dolt schemas should use "type" instead of "kind"

Current output from dolt table import --help:

Schema definition files are json files in the format:
{
"fields": [
{"name":"FIELD_NAME", "kind":"KIND", "Required":[true|false]},
...
],
"constraints": [
{"constraint_type":"primary_key", "field_indices":[INTEGER_FIELD_INDEX]}
]
}

     where "fields" is the array of columns in each row of the table
"constraints" is a list of table constraints.  (Only primary_key constraint types are supported currently)
FIELD_NAME is the name of a column in a row and can be any valid string
KIND must be a supported noms kind (bool, string, uuid, uint, int, float)

"kind" is a noms concept, not a database concept. We should use "type" or "datatype" everywhere.

Dolt: Schema diff no worky for changing primary keys

timsehn:videogame-sales timsehn$ dolt diff
diff --dolt a/videogame_sales b/videogame_sales
--- a/videogame_sales @ q1s2plh3ans4d9drcei5gi4futgmovhu
+++ b/videogame_sales @ gl5087ols1lji8csbom2hfj3fall5ich
  CREATE TABLE videogame_sales (
<   Rank int comment 'tag:0'
>   Rank int comment 'tag:0'
<   Name varchar comment 'tag:1'
>   Name varchar comment 'tag:1'
<   Platform varchar comment 'tag:2'
>   Platform varchar comment 'tag:2'
    Year int comment 'tag:3'
    Genre varchar comment 'tag:4'
    Publisher varchar comment 'tag:5'
    NA_Sales float comment 'tag:6'
    EU_Sales float comment 'tag:7'
    JP_Sales float comment 'tag:8'
    Other_Sales float comment 'tag:9'
    Global_Sales float comment 'tag:10'
  );

Failed to merge schemas
timsehn:videogame-sales timsehn$ dolt schema
videogame_sales @ working
CREATE TABLE videogame_sales (
  Rank int comment 'tag:0',
  Name varchar comment 'tag:1',
  Platform varchar comment 'tag:2',
  Year int comment 'tag:3',
  Genre varchar comment 'tag:4',
  Publisher varchar comment 'tag:5',
  NA_Sales float comment 'tag:6',
  EU_Sales float comment 'tag:7',
  JP_Sales float comment 'tag:8',
  Other_Sales float comment 'tag:9',
  Global_Sales float comment 'tag:10',
  primary key (Name,Platform)
);

timsehn:videogame-sales timsehn$ dolt schema HEAD^
videogame_sales @ HEAD^
CREATE TABLE videogame_sales (
  Rank varchar comment 'tag:0',
  Name varchar comment 'tag:1',
  Platform varchar comment 'tag:2',
  Year varchar comment 'tag:3',
  Genre varchar comment 'tag:4',
  Publisher varchar comment 'tag:5',
  NA_Sales varchar comment 'tag:6',
  EU_Sales varchar comment 'tag:7',
  JP_Sales varchar comment 'tag:8',
  Other_Sales varchar comment 'tag:9',
  Global_Sales varchar comment 'tag:10',
  primary key (Rank)
);

I'll create a failing bats test for this.

dolt: dolt config --global --add currently succeeds

timsehn:bats-test timsehn$ dolt config --global --add 
Config successfully updated.
timsehn:bats-test timsehn$ dolt config --global --list
timsehn:bats-test timsehn$

This should throw a wrong number or arguments error.

There is a skipped bats test for this.

documentation/errors regarding --schema <file>

  1. Typo in description of schema file (dolt table import --schema)

{ "fields:" [ {"name":"FIELD_NAME", "kind":"KIND", "Required":[true|false]}, ... ], "constraints": [ {"constraint_type":"primary_key", "field_indices":[INTEGER_FIELD_INDEX]} ] }

"fields:" should be "fields":

  1. Error when data doesn't match field type
    For example, I had the primary key field kind as "int", and when I tried to import a row that was a range "1901-2000", I got the following error:
    An error occurred moving data. Mapping transformfailed. Bad Row
    Can this error message be improved, and/or suggest using --continue to skip rows that aren't the correct datatype?

  2. I suggest adding more user friendly error messaging for when brackets are added/omitted incorrectly to the schema file. The "Required" field does not require brackets but the "field_indices" field does, for example:

{ "fields": [ {"name":"Year", "kind":"int", "Required":true}, {"name":"Punxsutawney Phil", "kind":"string", "Required":false}, {"name":"February Average Temperature", "kind":"float", "Required":false}, {"name":"February Average Temperature (Northeast)", "kind":"float", "Required":false}, {"name":"February Average Temperature (Midwest)", "kind":"float", "Required":false}, {"name":"February Average Temperature (Pennsylvania)", "kind":"float", "Required":false}, {"name":"March Average Temperature", "kind":"float", "Required":false}, {"name":"March Average Temperature (Northeast)", "kind":"float", "Required":false}, {"name":"March Average Temperature (Midwest)", "kind":"float", "Required":false}, {"name":"March Average Temperature (Pennsylvania)", "kind":"float", "Required":false} ], "constraints": [ {"constraint_type":"primary_key", "field_indices":[0]} ] }

When I mistakenly added brackets to the Required field, I received this error message:
When attempting to move data from csv file:groundhog_data_master.csv to dolt table:groundhog3, could not determine the output schema. Schema File: "schema.json" explicit pk: "" cause: json: cannot unmarshal array into Go struct field jsonFieldData.required of type bool

And when omitting them for "field_indices", I received this error:
When attempting to move data from csv file:groundhog_data_master.csv to dolt table:groundhog3, could not determine the output schema. Schema File: "schema.json" explicit pk: "" cause: json: cannot unmarshal number into Go struct field jsonConstraint.field_indices of type []int

Dolt: Feature Request: dolt table import -c --pk=auto

This should just make assign an ascending integer primary key for each record that's imported on a newly created table. We're going to get a lot of tables that don't have a primary key and this could be a convenience so you don't have to do parsing before importing into dolt.

Dolt: `dolt status` panics on an empty repo

$ dolt status
panic: error: Unable to read HEAD commit root value

goroutine 1 [running]:
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/commands.Status(0xc000037790, 0xb, 0xc00000e070, 0x0, 0x0, 0xc00009cf00, 0x6)
	/Users/mjesuele/dev/ld/dolt/go/cmd/dolt/commands/status.go:34 +0x276
github.com/liquidata-inc/ld/dolt/go/cmd/dolt/cli.GenSubCommandHandler.func1(0x1d766ae, 0x4, 0xc00000e070, 0x1, 0x1, 0xc00009cf00, 0xe)
	/Users/mjesuele/dev/ld/dolt/go/cmd/dolt/cli/command.go:61 +0x1ac
main.runMain(0x0)
	/Users/mjesuele/dev/ld/dolt/go/cmd/dolt/dolt.go:95 +0x2e0
main.main()
	/Users/mjesuele/dev/ld/dolt/go/cmd/dolt/dolt.go:56 +0x22

Not terrible, but should be more like the output of dolt log:

$ dolt log
Fatal error: cannot get HEAD commit for current branch.

Dolt: enforce SQL table naming conventions in dolt table create

timsehn:bats-test timsehn$ dolt table create -s=1pk5col.schema 1pk
timsehn:bats-test timsehn$ dolt table create -s=2pk5col.schema 2pk
timsehn:bats-test timsehn$ dolt schema
1pk @ working
CREATE TABLE 1pk (
  pk int not null comment 'tag:0',
  c1 int comment 'tag:1',
  c2 int comment 'tag:2',
  c3 int comment 'tag:3',
  c4 int comment 'tag:4',
  c5 int comment 'tag:5',
  primary key (pk)
);

2pk @ working
CREATE TABLE 2pk (
  pk1 int not null comment 'tag:0',
  pk2 int not null comment 'tag:1',
  c1 int comment 'tag:2',
  c2 int comment 'tag:3',
  c3 int comment 'tag:4',
  c4 int comment 'tag:5',
  c5 int comment 'tag:6',
  primary key (pk1,pk2)
);

tables cannot start with digits.

I'm going to create table naming bats tests.

Should Diffs handle schema changes better

I changed the id column from a string to an int and reimported the same data. This is the diff output:

timsehn:dolt-video timsehn$ dolt diff
diff --dolt a/employees b/employees
--- a/employees @ 3suru3esi5qtd6or5e61127agr1e55s6
+++ b/employees @ 32t16pjc0112e7q2vpql8j4lpvqve0rl
   id | first name | last name | title            
 - 0  | tim        | sehn      | ceo              
 - 1  | aaron      | son       | founder          
 - 2  | brian      | hendriks  | founder          
 - 3  | matt       | jesuele   | software engineer
 + 0  | tim        | sehn      | ceo              
 + 1  | aaron      | son       | founder          
 + 2  | brian      | hendriks  | founder          
 + 3  | matt       | jesuele   | software engineer

I guess this is technically correct. I changed the meaning of the row so it thinks I removed and added the rows.

Dolt: Undetected Schema merge conflict

See tim-bug branch

I removed a column checked it in. Went to push to master. Master was ahead. Pulled down a table that still had the column. No merge conflict. No table is busted.

timsehn:lunch-places timsehn$ dolt fetch origin master
Counted chunks: 9, Buffered chunks: 5)
timsehn:lunch-places timsehn$ dolt merge remotes/origin/master
Updating gjio4j5hk58ce1mqpavg65cot43j3bmu..5tipunvdh9j6s9bmfkvfsv2bl47pjod1
lunch-places | 2 ++
1 tables changed, 2 rows added(+), 0 rows modified(*), 0 rows deleted(-)
timsehn:lunch-places timsehn$ dolt diff
diff --dolt a/lunch-places b/lunch-places
--- a/lunch-places @ 2f3bscpsg8e0snhgc8a6tj8anhjjnpd9
+++ b/lunch-places @ 1t3huhuhom9u2nf49puhs4l4qqd85bi4
   name       | type of food  | rating | best dish        | description of location                   | visitors             
 + Bibibop    | asian noodles | 7      | noodle bowl      | south east corner of 3rd and santa monica | aaron,daylon         
 + Sake House | sushi         | 85     | King Salmon Roll | Downstairs of the office                  | zach,aaron,brian,matt
timsehn:lunch-places timsehn$ dolt table select lunch-places
error: failed to transform row name:"Art's Table" | type of food:"fancy" | rating:8 | description of location:"Montana and 9th. Near Tim's house" | best dish:"short rib press".
name      |type of food |rating|best dish       |description of location                  |visitors             
Bibibop   |asian noodles|7     |noodle bowl     |south east corner of 3rd and santa monica|aaron,daylon         
Sake House|sushi        |85    |King Salmon Roll|Downstairs of the office                 |zach,aaron,brian,matt

Weird dolt import bug

Table

timsehn:dolt-video timsehn$ cat /tmp/employees.csv 
id,first name,last name,title
0,tim,sehn,ceo
1,aaron,son,founder
2,brian,hendriks,founder
3,matt,jesuele,software engineer

Schema

timsehn:dolt-video timsehn$ cat /tmp/employees.schema 
{
  "fields": [
    {
      "name": "id",
      "kind": "number",
      "required": true
    },
    {
      "name": "first name",
      "kind": "string",
      "required": false
    },
    {
      "name": "last name",
      "kind": "string",
      "required": false
    },
    {
      "name": "title",
      "kind": "string",
      "required": false
    }
  ],
  "constraints": [
    {
      "constraint_type": "primary_key",
      "field_indices": [
        0
      ]
    }
  ]
}

Output of import:

timsehn:dolt-video timsehn$ dolt table import -c -f employees /tmp/employees.csv -s /tp/employees.schema 
A bad row was encountered while moving data.
Bad Row:id:"3" | first name:"matt" | last name:"jesuele" | title:"software engineer"
strconv.ParseBool: parsing "3": invalid syntax
These can be ignored using the '--continue'

Tried with the id set to integer as well. Not sure what types are supported.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.