Git Product home page Git Product logo

data's Introduction

Github: .github/workflows/sync-to-gitlab.yml โ†’ to synchronise/backup this repo to gitlab

Gitlab: pipelines โ†’ to deploy profiles.joblist.today

Data

A list of projects and companies potentially hirring, editable by everyone in full transparency with open authoring, each updates submitted through git(hub).

Check out the licenses at the bottom of this page for usages and contributions

This project aims to be a usefull ressource, and to be used as data base for ways to search and explore the data (ex : https://joblist.today)

All data in this repository is used a data source for all other joblist.today projects (websites, search etc.). All welcome to use and contribute data, ideas, prototypes...

Notes on contributing

Everyone is welcome to contribute to this project and to all data.

Try to be objective and informative when editing content, so the information is clear and usefull for everyone.

Also, feel free to discuss anything in a Pull Request or Issue.

The data models should evolve into a more usefull representation of a project/company available data.

How to use the project

In addition to explore and editting the content of this repository (it is folders and files) manually (in git/github), it is also possible to use:

  • edit.joblist.today to add a new company/projects and edit existing ones. The other way is by making a PR to this repository, editting any part of the content.
  • profiles.joblist.today to explore the content of this project, companies data.
  • joblist.today to search all the jobs that can automatically be indexed

Other ressources

How to use netlify-cms & github authentication

Netlify CMS (content management system) is the most straight forward way to contribute to this project, and anyone is allowed to contribute.

By login-in with your Github account, it will provide a graphical interface to edit and review the data.

  1. Login at edit.joblist.today/cms, with your free Github account.
  2. add new content or edit existing one through the interface
  3. save your changes. They will be stored as draft
  4. go to the workflow tab of the CMS, and move (drag) your changes to the In Review section.
  5. administrators & contributors will discuss and review your changes (visible as Pull Requests on github@joblisttoday/companies/pulls)
  6. your pull request is then merged (or closed), and your github account notified
  7. changes will be visible live on the profiles.joblist.today site (can take 1/2 hour; see gitlab-ci:sync and build)

Netlify-CMS (in open-authoring mode) will handle the work to add a contribution to this project; to create/edit the data models, each companies, so you everyone can safely submit changes.

The code for the netlify-cms instance is hosted here: gitlab@joblist/cms).

Notes on job board providers

Job list today systems are able to automatically display and index jobs from a variety of job board providers.

Test and learn more about which providers are currently available, check providers.joblist.today

it is possible to use the providers as javascript web-components (to embed them in any website), and also in node through their simplified API to the job board providers (see the job list workers code).

only providers that do not require authentication are currently supported; checkout known issues (or create one)

Project's folder structure

The folders of this project/repository are as follow:

  • In ./companies, all companies that might be hirring
  • In ./tags, all tags used to describe companies (and cities), for which we have information; can be used for featured tags etc.
  • In./cities, all cities/region that we want to feature, because a certain number of companies are hirring there

Deployment

Look in the ./gitlab-ci.yml file (the github project made for contributions, synchronises to gitlab, which runs the CI pipelines).

The project is automatically synchronised to https://gitlab.com/joblist/data, which runs the gitlab-ci.yml CI/CD pipelines.

This repository on Github is only used because netlify-cms open-authoring mode requires a repository hosted on Github ( netlifcy-cms github backend, > editorial-workflow mode & graphql-api activated).

Development

There are no specific development steps. If you use the data from this repo, remember the keys of the data might evolve, so pin your project to a commit of this repository (and upgrade safely).

This folder aims to be data only, with all data represented as files inside folders.

The data files currently have the .md extension as their content is markdown, and the data is stored in the frontmatter part of the file (the part always at the top of the file, between the two ---).

For an example on how to parse the data with javascript, check out the job list workers: https://gitlab.com/joblist/workers/-/blob/main/src/database.js

Licenses

  • code: GPLv3
  • data & contributions: Creative Commons: Attribution + Noncommercial + ShareAlike (BY-NC-SA) <- to the joblist contributors.

data's People

Contributors

0x137f avatar 4www avatar hugurp avatar lperreau avatar oskarrough avatar rmnprkrl avatar shochdoerfer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

data's Issues

convert data from markdown to json

Because markdown is so annoying to work with:

  • need a library to convert frontmatter and content to JSON (or HTML)
  • all libraries are wacky, good, but you never know wich one to use
  • writting markdown frontmatter config is hard, even more when JSON object are stringified (positions.[].map)

Therefore JSON will be used:

  • hard to writte by hand, but easier than markdown
  • all data in this repo should be consise, company.description does not need rich text, or font styles
  • can/should use the cms for edition

Converted

Improve `tags` collection

Right now we use tags:

  • on company.tags, to describe companies, it is a list of "free text field"
  • as tags collection, but not currently used anywhere except the /tags/:tag page if it exist, to add data to a tag

But the tags are overall not so great, as a user experience.

  • their semantic is unclear
  • it is not really grouping companies together
  • hard to figure what tag to add / remove, or what is available

We could improve tags:

  • enforce companies.tags to pick from the tags collection, (create a relation between the two models, in the cms)
  • clearly define the source of the tags collection list. Why does a tag exist, what value does it bring to the user

The tags need to evolve to be able to well describe a company.

Maybe it should be additional categories on the company model.

Examples:

All in all this seems like a complex project, that needs to bring clear and accessible semantics, to add value for the end user when exploring companies and looking for jobs.

(related cms issue https://gitlab.com/joblist/cms/-/issues/4)

Clean up company data attributes and uniformize them

Currently the data in this repo, for companies, has issues, for examples with dates.

That leads to having the sqlite3 database tables to be generated with for example dates (created_at and updated_at) in different formats (cleanup here, in post processing https://gitlab.com/joblist/workers/-/merge_requests/2).

We should fix the data in two places:

  • in the cms, to be sure that the data is created by the CMS (and user), in the correct format https://gitlab.com/joblist/cms
  • in the data repo, to cleanup existing data in the correct format (sent also by the CMS)

Github action: check if company exists

Create a Github action/workflow, that will check if a company already exists in this repository's data.

If the check passes, the PR is allowed to be merged

If the check fails, a company exists, and the PR needs further review

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.