Git Product home page Git Product logo

dlai's Introduction

This is a repo for the DLAI squad

In this repository you will find the scripts we are currently using to build out the corpus, create the translator, and generate results for the manuscript.

Each folder has it's own README and the titles of the folders should be self-explanatory.

Please note that the datasets i.e. UMLS are not included here and some of them require a license. Contact me if you need access to any of these.

Additionally, since prodigy is mostly being run on the command line, there isn't much here for that. The script that calls the models we are building is in /translate/automated_translator_v3.ipynb
To see the exact environment I built to run these scrips see the file 'DLAI_environment.yaml'
That file is quite robust and probs would be difficult to install on it's own so the most important packages are:

This requires conda.
We use the most current versions for each as of Oct 2021, I would first install rapidsai via the generated command on their website.
Then install CWI, dask, spaCy and scispaCy
Any dependencies after installing these can be safely installed, first try with conda, if not available then pip.

dlai's People

Contributors

kswanjitsu avatar

Stargazers

An Nguyen avatar

Watchers

 avatar  avatar

dlai's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.