514-labs / moose Goto Github PK
View Code? Open in Web Editor NEWThe developer framework for your data & analytics stack
Home Page: https://www.moosejs.com
License: MIT License
The developer framework for your data & analytics stack
Home Page: https://www.moosejs.com
License: MIT License
From SyncLinear.com | POL-84
null
From SyncLinear.com | 514-278
From SyncLinear.com | 514-275
From SyncLinear.com | 514-272
From SyncLinear.com | 514-268
From SyncLinear.com | 514-266
From SyncLinear.com | POL-80
When I spin up dev
I get 1/2 times:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Network(hyper::Error(Io, Os { code: 54, kind: ConnectionReset, message: "Connection reset by peer" }))', src/infrastructure/console.rs:38:81
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[1] 99364 abort ../igloo-stack/apps/igloo-kit-cli/target/debug/igloo-cli dev
Looks like we need to be more tolerant to network resets
I'd use indicatif and console to accomplish this
From SyncLinear.com | POL-52
From SyncLinear.com | POL-81
We need to be able to use the framework to instrument igloo itself and iterate on the data collected
From SyncLinear.com | POL-109
From SyncLinear.com | POL-105
From SyncLinear.com | POL-79
./src/app/constants.ts:1:1
#21 19.10 moose-console:build: Type error: 'constants.ts' cannot be compiled under '--isolatedModules' because it is considered a global script file. Add an import, export, or an empty 'export {}' statement to make it a module.
#21 19.10 moose-console:build:
#21 19.10 moose-console:build: > 1 |
#21 19.10 moose-console:build: | ^
#21 19.15 moose-console:build: ELIFECYCLE Command failed with exit code 1.
#21 19.17 moose-console:build: ERROR: command finished with error: command (/app/apps/moose-console) /usr/local/bin/pnpm run build exited (1)
#21 19.18 moose-console#build: command (/app/apps/moose-console) /usr/local/bin/pnpm run build exited (1)
And Add some CI to make sure we don't break this inadventendly
From SyncLinear.com | POL-58
From SyncLinear.com | POL-64
From SyncLinear.com | POL-56
From SyncLinear.com | POL-89
This should probably include defining a schema for the red panda schema registry to enable data quality enforcement. As I've dug into this it seems like protobufs will be a requirement to be able to enforce schemas in the registry. This is something we've talked about in the past. Given that we're hoping for multi language support, it may make sense to leverage ~~~~~~protobufs~~~~~~ avro as the cross-language serialization it may make sense to make avro easy to work with.
From SyncLinear.com | POL-72
From SyncLinear.com | POL-72
We've explored leveraging an ORM to create the needed abstraction for the ingestion point. We believe that an ORM DSL based approach, like Prisma Schema Language is the best past forward where the ingestion point would take in a ORM generated type as it's input and enable the user to easily create an ingestion point.
With that in mind, though, we'd like to enable users to create ingestion points quickly. To do so, we will support creating an ingestion point and passing it a typescript interface as a generic type or function parameter (approach TBD).
The hope is for the user to be able to run this command and then have the ingestion point configured for them. We need to support:
$ igloo add ingestion-point --name=user-event -a=timestamp:utc -a=packet:json -a=type:enum
The hardest problem will be enabling the developer to write code against typescript files and automatically update the migration. Alternatively, we can force the developer to define their parameters through the CLI or to manually update their migration scripts if they decide to write parameters manually.
From SyncLinear.com | POL-78
Create a function to create readable and consistent CLI messages as opposed to one-off print statements
From SyncLinear.com | POL-51
From SyncLinear.com | POL-51
From SyncLinear.com | POL-55
I'm trying to flesh out whether we should create an ORM as part of the igloo framework. It seems like there would benefits both for the developer experience when using the framework but could also serve as an abstraction layer over different databases and stream systems.
It could then integrate nicely into the broader flows paradigms and possibly be used in conjunction with orchestration frameworks.
After exploring this topic we're concluding that contibuting a clickhouse connector to Primsa is like the best path forward. We're excited about the way they do migrations. We'll need to investigate more to understand the LOE here but from the preliminary investigation it seems feasible.
From what I can gather, we'll need DMMF support, type mapping for clickhouse supported types and likely some tweaks to the migrations given that alter statements are different in clickhouse than they are in other systems
From SyncLinear.com | POL-76
From SyncLinear.com | 514-192
From SyncLinear.com | POL-57
From SyncLinear.com | POL-54
Right now we have a series of docker run command with a network setup in between.
We are managing all that and making the glue between the different element when we could generate a docker-compose.yaml
file in the project directory and use all the optimizations from there when spinning up containers.
docker-compose
is shipped with docker, so that should not add another dependency.
That should simplify our code and reduce the number of bugs.
Latest bug I spotted is that if the config of a container changes through a new version of the CLI, the containers fail to boot because we are only creating containers once and then running them. But if the configuration changes, we are not updating it. This is logic that is embedded in docker-compose
sfully validated clickhouse docker container
Failed to run console container: docker: Error response from daemon: Conflict. The container name "/console-1" is already in use by container "b2cdf08f387864fb94d8546a5767391ef08dd89f7b5813afc51e685d1ff129f6". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
Failed to run console container: docker: Error response from daemon: Conflict. The container name "/console-1" is already in use by container "b2cdf08f387864fb94d8546a5767391ef08dd89f7b5813afc51e685d1ff129f6". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
Failed to validate red panda cluster
From SyncLinear.com | POL-68
null
From SyncLinear.com | 514-274
From SyncLinear.com | 514-270
From SyncLinear.com | 514-267
From SyncLinear.com | 514-265
From SyncLinear.com | POL-67
The webserserver serves up ingestion points from the app/ingestion-points directory
From SyncLinear.com | POL-86
From SyncLinear.com | POL-50
From SyncLinear.com | POL-83
From SyncLinear.com | POL-59
From SyncLinear.com | POL-69
From SyncLinear.com | POL-60
null
From SyncLinear.com | 514-274
From SyncLinear.com | 514-270
From SyncLinear.com | 514-267
From SyncLinear.com | 514-265
From SyncLinear.com | 514-263
From SyncLinear.com | 514-225
From SyncLinear.com | POL-53
null
From SyncLinear.com | 514-278
From SyncLinear.com | 514-275
From SyncLinear.com | 514-272
From SyncLinear.com | 514-268
From SyncLinear.com | POL-66
From SyncLinear.com | POL-74
We need to be able to use the framework to instrument igloo itself and iterate on the data collected
From SyncLinear.com | POL-109
From SyncLinear.com | POL-82
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.