Git Product home page Git Product logo

startup-os's Introduction

StartupOS

Gitter CircleCI

Examples for Google's Open Source stack and deployment to the cloud.

The main technologies in the stack are:

Hands-on experience

Try these Google Cloud Shell tutorials:

Multi-language code formatting tool built with Bazel:

Open in Cloud Shell

Field & enum renaming API backwards compatibility 1: how old data can still be read by new release, even after renaming:

Open in Cloud Shell

Field & enum renaming API backwards compatibility 2: how old releases can talk to new releases without breaking:

TBD

Supported languages

Protos, gRPC and Bazel are polyglot. The examples in this repo are mostly in Java and Typescript, but there's support for many other languages:

  • gRPC and Protocol Buffers are supported by Google in C++, Java (and Android Java), Python, Go, C#, Objective-C, PHP, Dart, Ruby and JavaScript (incl. Node.js).
  • Bazel is supported by Google in Java (incl. Android builds), Objective-C (incl. iOS builds), C++, Go, Dart, Rust, Sass and Scala.
  • The community has added support for many others languages. See this list for Protocol Buffers and gRPC, and this one for Bazel.

Top examples

  • Lots of Protocol Buffer examples.
  • gRPC-Web: a js client library running in the browser, connected to a gRPC server through an HTTP proxy. Both server and client use gRPC auto-generated stubs to handle communication.
  • gRPC microservices example.
  • Docker example: building containers using Bazel (no dockerfile needed!).
  • Kubernetes: a config file showing how to run a container built with Bazel on k8s.
  • CI example: Using CircleCI to run CI (continuous integration) and test all Bazel targets.
  • Firebase: Java client for storing Protocol Buffers here.
  • Dagger: Java Dependency Injection framework ("Next gen Guice"), see examples here.
  • Flogger: Java logger with fluent API, see examples here.
  • Android app: An Android app built with Bazel and integrated to Firebase.

Tools

There are several useful tools in the tools section.

How to use StartupOS

You can treat StartupOS as a "developer image" with a pre-built setup and associated tools.

You can either:

Installation

Install Bazel. That's it!

Build & Test

  • Build everything: ./build.sh
  • Run all tests: ./test.sh

About monorepos

A monorepo is a software development approach where all code is stored in a single repository. Some things are easier to do in a monorepo, such as sharing a proto file across front-end and backend, some things are harder, such as per-repo control over collaborators, email notifications, commit history etc. We're working on an approach where multiple repos can act as a single monorepo, while still being separate repos, using multi-repo tooling such as multi-repo code review.

Some good reads about the monorepo approach:

Platforms

While Bazel supports Linux, Mac and Windows, this repo supports Linux and Mac. If you're on Windows 10, you can use Windows Subsystem for Linux (WSL). It gives you a Linux environment, without the overhead of a virtual machine.

Installing Windows Subsystem for Linux:

Please follow the guide. Note: You may have to do a Windows upgrade. If you need it, it will ask for it at the beginning of the installation.

Contributing

You're welcome to contribute and in doing so, learn these technologies. You can have a look at the issues list, or at the project milestones.

startup-os's People

Contributors

a3l5o7n9 avatar elharo avatar gulilo avatar jdban avatar jin avatar lewish avatar lloydchang avatar matemahinkhan avatar nikleshphabiani avatar nlewycky avatar oferb avatar val-fed avatar vmax avatar webdevelopland avatar wilder avatar yanivsn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

startup-os's Issues

Add dependencies.yaml formatting to multi-language formatter

Essentially this command:
bazel run @bazel_deps//:parse -- format-deps --overwrite --deps pwd/dependencies.yaml
Should be run as part of SimpleFormatter, possibly using a flag (unless @bazel_deps//:parse uses a standard formatter for yaml files).

Add yapf to formatter

//tools:formatter currently relies on yapf being available on the user's system.
We should:

  1. Make an http_archive that will make bazel download it
  2. Add it to a BUILD file as a binary target, i.e py_binary
  3. Add it to runtime_deps of //tools:formatter, so that it's available for the tool to use, and make the tool call it.

It's similar to how buildifier and clang-format work with the formatter.

Yapf: https://github.com/google/yapf/releases

Single directory for testing.

Hi,

I would like to suggest having some kind of a centralized place directory which will hold all the tests. This is order to separate tests from code, and having an easy way of finding one's class tests.

The suggested idea is as such:

Have a test directory at the root of the project, on each added class have a corresponding test class under the test directory, where the path underneath the test class will correspond to the path of the original class.

For example, lets say we have the following project:

startupos/
  code_reviewer/
    some_feature/
      SomeInteface.java
      SomeImpl.java

the matching test directory will look as such:

startupos/
  tests/
    code_reviewer/
      some_feature/
        SomeImplTest.java

What do you think?

Make `web_login` ask for the same permissions as `Reviewer`

Right now, web_login asks for more permissions than needed, and we should only ask for those that Reviewer asks for, which are the standard Firebase auth permissions.

We are currently asking for "View and manage your data across Google Cloud Platform services".

Tools folder needs more documentation

Looking in the tools folder, there seem to be some interesting tools, but there is no documentation. What is aa? What is buildtools_wrapper? Ideally, we don't have to hunt through the source code to find out, so some documentation would be great.

Ability to choose exact commits in Reviewer text diff

  • Each code block in file-changes should be able to be picked with exact commit (not only base and last)
  • Possible UI implementation: drop-down list.
  • Url should store left and right state.
  • Clicking on a comment would open up the text diff. It will be on the right, if it's last commit, or on the left, if there was some updates, after the comment.

#333

Highlight locally modified files in Reviewer

Make UI highlight these files:

  • Locally modified, uncommitted files
  • Committed but not yet pushed - a boolean field should be added to notify the UI. We will later set that field correctly in the local server.

Unable to run `bazel query //...` on StartupOS

I'm not sure where else to report this bug I've encountered, so if it isn't the right place, would someone from the StartupOS team kindly point me in the right direction?

Problem

After forking and cloning StartupOS to my Windows computer, if I try to run bazel query //... (part of the installation instructions), I encounter the following error:

ERROR: error loading package '': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_typescript//': No WORKSPACE file found in C:/users/jonathan/_bazel_jonathan/zjfrytj2/external/build_bazel_rules_typescript

When I navigate to C:/users/jonathan/_bazel_jonathan/zjfrytj2/external/build_bazel_rules_typescript, Windows throws up the following error message box:

image

How to reproduce

  1. git clone https://github.com/jbduncan/startup-os
  2. cd startup-os
  3. bazel query //...

Environment

Operating System

Windows 7

bazel version

Build label: 0.15.0
Build target: bazel-out/x64_windows-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 26 12:09:42 2018 (1530014982)
Build timestamp: 1530014982
Build timestamp as int: 1530014982

Folder structure for packages and Java

Haven't tried to build yet, but I was struck that some of the java code for aa and LocalServer isn't organized along the usual package conventions, though the code itself does have package statements. E.g. in AaTool.java

Local server exception while opening new file

If file was created with one of diff commits, opening the file throws the exception:

fatal: Path 'tools/local_server/my-new-file.txt' does not exist in 'f7be0169da78ead679e7491e726ed481532a0336'

	at com.google.startupos.common.repo.GitRepo.runCommand(GitRepo.java:95)
	at com.google.startupos.common.repo.GitRepo.runCommand(GitRepo.java:78)
	at com.google.startupos.common.repo.GitRepo.runCommand(GitRepo.java:70)
	at com.google.startupos.common.repo.GitRepo.getFileContents(GitRepo.java:350)
	at com.google.startupos.tools.reviewer.localserver.service.CodeReviewService.readTextFile(CodeReviewService.java:108)
	at com.google.startupos.tools.reviewer.localserver.service.CodeReviewService.readTextFile(CodeReviewService.java:138)
	at com.google.startupos.tools.reviewer.localserver.service.CodeReviewService.getTextDiff(CodeReviewService.java:206)
	at com.google.startupos.tools.reviewer.localserver.service.CodeReviewServiceGrpc$MethodHandlers.invoke(CodeReviewServiceGrpc.java:697)
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:706)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Example: https://startupos-5f279.firebaseapp.com/diff/65/startup-os/tools/local_server/my-new-file.txt

Thinking task: multi-repo CI

Now that we have a basic multi-repo cli, multi-repo Code Review, think about how we can do multi-repo CI. Say we have startup-os and hasadna as the 2 repos. Author presses submit after approval in Reviewer, to a diff that changes both repos. Now what do we do to push the changes to both repos, and make hasadna point to the new commit in startup-os?

Fix placeholder lines returned from LocalServer

Local server marks lines as changed not by them actual number, but by number after adding placeholders (blue numbers on the pic).
image
Line 27 isn't highlighted, because placeholder 31 is highlighted instead.
Temporary front-end method, which fixes the issue:

tempFixChangesLineNumber(textChanges: TextChange[]): void {
  let delimiter: number = 0;
  textChanges.forEach(textChange => {
    switch (textChange.getType()) {
      case ChangeType.DELETE:
      case ChangeType.ADD:
        textChange.setLineNumber(textChange.getLineNumber() - delimiter);
        break;
      case ChangeType.LINE_PLACEHOLDER:
        delimiter++;
    }
  });
}

Add command: aa killserver

The command essentially does this:

# Kill previous:
if [ "$(uname)" = "Darwin" ] || [ "$(uname)" = "Linux" ]; then
  # Mac & Linux
  # kill HTTP gateway (port 7000), gRPC server (port 8001) and Angular server (8000)
  # -t makes `lsof` output only PIDs so output can be piped to `kill`
  # -n and -P prevent `lsof` from resolving addresses and ports, therefore
  # making execution faster
  # we filter for processes that listen on port in order to kill only
  # servers (i.e. Angular) instead of clients (i.e. Chrome)
  kill $(lsof -tnP -i:7000 -i:8000 -i:8001 -sTCP:LISTEN)
# else
#   # Windows
#   # TODO: kill local_http_gateway & local_server
fi

Reviewer UI/UX changes

  • Comments summary in diff page - including general Diff comments.
  • Comments in text diff page - including "Resolve" button.
  • Delete Diff button
  • "Request Review" button.

Add yapf and clang-format to formatter

//tools:formatter currently relies on yapf and clang-format being available on the user's system.
We should make a BUILD target that will make bazel download them, and then add them to runtime_deps of //tools:formatter, so that they're available to the tool to use.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.