Git Product home page Git Product logo

aide's Introduction

aguaclara

Pypi Version Documentation Build Status Code Coverage

aguaclara is a Python package developed by AguaClara Cornell and AguaClara Reach for designing and performing research on AguaClara water treatment plants. The package has several main functionalities:

  • DESIGN of AguaClara water treatment plant components
  • MODELING of physical, chemical, and hydraulic processes in water treatment
  • PLANNING of experimental setup for water treatment research
  • ANALYSIS of data collected by ProCoDA (process control and data acquisition tool)

Installing

The aguaclara package can be installed from Pypi by running the following command in the command line:

pip install aguaclara

To upgrade an existing installation, run

pip install aguaclara --upgrade

Using aguaclara

aguaclara's main functionalities come from several sub-packages.

  1. Core: fundamental physical, chemical, and hydraulic functions and values
  2. Design: modules for designing components of an AguaClara water treatment plant
  3. Research: modules for process modeling, experimental design, and data analysis in AguaClara research

To use aguaclara's registry of scientific units (based on the Pint package), use from aguaclara.core.units import u. Any other function or value in a sub-package can be accessed by importing the package itself:

Example Usage: Design

import aguaclara as ac
from aguaclara.core.units import u

# Design a water treatment plant
plant = ac.Plant(
    q = 40 * u.L / u.s,
    cdc = ac.CDC(coag_type = 'pacl'),
    floc = ac.Flocculator(hl = 40 * u.cm),
    sed = ac.Sedimentor(temp = 20 * u.degC),
    filter = ac.Filter(q = 20 * u.L / u.s)
)

Example Usage: Core

# continued from Example Usage: Design

# Model physical, chemical, and hydraulic properties 
cdc = plant.cdc
coag_tube_reynolds_number = ac.re_pipe(
    FlowRate = cdc.coag_q_max,
    Diam = cdc.coag_tube_id,
    Nu = cdc.coag_nu(cdc.coag_stock_conc, cdc.coag_type)
)

Example Usage: Research

import aguaclara as ac
from aguaclara.core.units import u
import matplotlib.pyplot as plt

# Plan a research experiment
reactor = ac.Variable_C_Stock(
    Q_sys = 2 * u.mL / u.s, 
    C_sys = 1.4 * u.mg / u.L, 
    Q_stock = 0.01 * u.mL / u.s
)
C_stock_PACl = reactor.C_stock()

# Visualize and analyze ProCoDA data
ac.iplot_columns(
    path = "https://raw.githubusercontent.com/AguaClara/team_resources/master/Data/datalog%206-14-2018.xls", 
    columns = [3, 4], 
    x_axis = 0
)
plt.ylabel("Turbidity (NTU)")
plt.xlabel("Time (hr)")
plt.legend(("Influent", "Effluent"))

The package is still undergoing rapid development. As it becomes more stable, a user guide will be written with more detailed tutorials. At the moment, you can find some more examples in specific pages of the API reference.

Contributing

Bug reports, features requests, documentation updates, and any other enhancements are welcome! To suggest a change, make an issue in the aguaclara Github repository.

To contribute to the package as a developer, refer to the Developer Guide.

aide's People

Contributors

eak24 avatar emmasung avatar fletchapin avatar iancullings avatar oliver-leung avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aide's Issues

Sed 0.6 to 6 L/s PST

  • apply PST all the way down
  • switch to Pipeline
  • get pipe dimensions from Pipeline Fittings Service
  • add floc hopper drain

Documentation Engine

One of the goals for AIDE is to add functionality for generating completed documentation on building and maintaining the plant. As per @monroews's request, we will be reviving the work of AIDE Document (and this repo) and tying it directly to the functionality of AIDE.

Documentation templates (from which completed documentation will be generated) will take on the form of RST files, due to RST's ability to render LaTeX equations. Hopefully, we will be able to add images from Onshape 3D models and insert them directly into these doc templates. Click here to download an example documentation template in Spanish.

Looking to the future, my vision is that the aguaclara package will remain as it is currently - a library of functions for water treatment research and design code. AIDE (which will reside in this repo) will become a GUI-based application (run either locally or in an online web application), and will be a nexus between the aguaclara design code, a documentation engine, and Onshape.

Pipe library and database integration

The feature script pipe database and the part studio pipe library should get their data from a single source. I really like the idea of having a pipe data base in a CSV file. That database could presumably be used to by the pipe library for drawing pipes and could be accessed by featurescript for getting dimensions needed for design. This would be more elegant than using superderive to access the pipe library to extract a pipe dimension.
Here is a nice data source https://www.gfps.com/country_US/en_US/lightbox/pvc-cpvc-tech/metric/metric-dimensions.html. The research IT team is currently working to create a pipe class in python. We could use a common data source for both python and FS.
There are two approaches to obtaining all of the pipe dimensions. One approach is to use equations to define the ID (based on SDR) and the other approach is to include the ID in the database. One reason to include the ID in the database is that the actual IDs are less than that calculated using SDR because the SDR specifies a minimum wall thickness. If we move to a CSV for the data source then the ID for SDR pipes should be in the csv file rather than calculated.

SuperDerive bugs

  1. Superderive throws errors when the geometry is suppressed and the mate connector is missing. Ideally this error would be eliminated since the mate connector isn't used if the geometry is suppressed.
  2. Many of our uses of superderive throw an error because there is a bad string in the "Configuration JSON or Map". This is because line 151 of SuperDerive is const configurationStringDefault = '\\\"{"my_parameter_id": "my_value"}\\\"'; Thus when that input isn't used it causes an errror.

Make an AIDE Guide/Tutorial

  • Short video just showing changing configurations (like flow rate) leading to change in model
  • Other supporting docs or a guide could be useful too

Fi: Onshape Drawing

  • backwash siphon pipe

  • valves for tee

  • update entrance tank

  • instancing/patterning

Cannot run F360 addins from other addins

@ethan92429 In order to implement your solution of having aide call aide_gui, we would have to have the F360 command (from the aide Add In) to run the palette_gui Add In, which would be nearly impossible.

So we have aide and palette_gui addin code folders within the API/AddIns folder on our computers, and within the aide AddIn code, we would have to import the palette_gui code to run the palette_gui AddIn. However, we can't do from .. import palette_gui (essentially going up two levels in the file system) because python doesn't allow it. Furthermore, we can't just put the palette_gui folder within the aide folder because F360 must run addins from the API/AddIns, not within any subfolders.

One workaround could be to hard code an absolute path to the palette_gui folder to import, but I want to avoid doing that.

Because of this, would it just be easier to have the user open up the palette_gui and then have the gui run all of aide's stuff?

Here is a forum post (from our good friend Marshal!) which says that it's impossible to run AddIns/Scripts from Scripts; I'm assuming that the same goes for running AddIns/Scripts from AddIns and their Commands.

Naming convention for variables

Given the standardized parameter names in this file, I'd like to propose a way to assign these names to both our F360 model parameters and our Design code.

Suppose we have the following (abbreviated) hierarchy within a design:

\floc                    #assembly
    \entTank             #subassembly
        \wall            #component
            thickness    #parameter

Within F360, we would label the parameter as each level of hierarchy, separated by underscores (floc_entTank_wall_thickness). That way, when we break the links when combining components into larger assemblies, we will always know exactly where each parameter came from.

Within Design, we could have the same kind of hierarchy with our folders and Python files. We could have a .py file fro each subassembly, and then each component/parameter would be a global variable (if constant), or a function otherwise (with this example, floc.entTank.wall.thickness).

This is a fair bit more verbose than our current naming conventions, and changing our current parameters would be no small task, but I think this would prevent us from ever needing to change our variable names due to using the same component/parameter name in different assemblies.

Pip install for dependency packages

Instead of leaving the entire source distributions (sdists) of the packages for our dependencies, we should instead be trying to run a local pip install on the user's computer. See this forum post for more information on our progress.

Create multi-level Smart Tank Template

This simple Smart Tank will serve as an example of how Smart Components are assembled and how inputs are passed to subcomponents. This entire file can be copied when starting new Smart Components or the code can be copied.

Add demonstration of multiple sub components being controlled by a plant level part studio.

Draw the mold used to make the diffusers

This is the angled mold that creates the slot. The mold should be drawn based on the sed tank design because the mold varies depending on the floc properties and temperature.

AIDE Summer Report #1

We've completed our first report for the summer - you can read it here.

You can also navigate to it in this repo, under reports/.

Integrate aide_render.render_lfom()

In the latest version of aide_render, I made a render_lfom() function that takes in an lfom yaml (right now supporting only the flow rate variable, Q) and returns the design YAML. Please incorporate this into the aide main loop. If you need to save the output as a yaml file, make sure to specify the yaml_path_out keyword argument.

To clarify, this is the function you'll be calling with the flow rate input. From there, this function will run and produce a design output. That design output will then be read by aide_draw. Please pip install aide_render and try to run the render_lfom() function. It should be well documented.

End Product Validation and Testing

Ensure the following tests are complete:

  • Unit tests of FS features
  • E2E test from the ACR website form to the documentation
    • Will probably have to be done manually

Plant level plumbing

  • floc to sed - done
  • sed to sump - done
  • ET to floc - done
  • chem plumbing
  • drain/waste plumbing

Combine aide_gui with aide

After working with the Fusion 360 API for a while, I still think that combining the functionality of aide and aide_gui would be the best way to integrate all of our modules. By that, I mean that aide_gui (which would reasonably be renamed to aide) will gather user inputs, then subsequently call aide_design/draw/document while passing those inputs as parameters. Combining the two would allow us to both avoid the shortcomings of the F360 API and more easily add functionality in the future.

Why aide can't call aide_gui

I ran minimal tests on as many of the following ideas as I could, but let me know if anything seems out of place and I'll retest it.

Returning user inputs

As is currently planned out in the ReadMe's MVP1, running aide_gui will return the location of the YAML containing the gathered user inputs. This can't be done for the following reasons:

  1. The user inputs can't be returned immediately when aide_gui is run because they don't yet exist. aide_gui would be pushed off of the call stack and aide would hand off an incomplete YAML to aide_design.
  2. The user inputs can't be returned when the user inputs their parameters because Python will get "impatient" and just return None if a return value isn't given immediately.

These problems stem from the fact that F360 add-ins use event handlers, so let's see what happens when we play around with them.

Event handling

Instead of having aide_gui return the user inputs, I tried having it trigger an event that would notify aide that the user inputs had been written to a YAML. As seen in aide.py and aide_gui.py, event handlers for add-ins are stored in a globally declared handlers list. In order for this to work, we'll need for aide and aide_gui to be able to "share" this list, either by having aide_gui return its handlers to aide, or by aide passing its handlers as a parameter when calling aide_gui.run.

Having aide_gui return its handlers won't work for similar reasons to why aide_gui can't simply return the location user inputs:

  1. The handlers can't be returned at the end of aide_gui.run because by that point, only one event handler (ShowPaletteCommandCreated) will be appended. There are two others that are needed for aide_gui to work (ShowPaletteCommandExecute and MyHTMLEvent), and with aide_gui.run pushed off of the call stack, there's no way to return those other handlers to aide.
  2. The handlers can't be returned after two or three handlers are appended because Python will get "impatient" and just return None.
  3. The notify functions for each event only act within the add-in where they're called (aide_gui in this case), and because they're called from the backend of F360, we can't manually call them from aide.

Having aide pass its handlers list to aide_gui would require the list to be passed it through each of the handlers' __init__(self) and notify(self, args) functions so that it gets appended to. This isn't possible, because the F360 backend expects for there to only be the provided self and args (in notify) parameters to be passed and throws an exception when more are given. Even if we managed to get all of the handlers successfully appended, there's still no way for us to get them back to aide for the above reasons.

Overall, the current system of aide being the master controller forces us to use the F360 API in a way that it's not designed for. There should only be one add-in running at a time, rather than an add-in within an add-in.

Why combining aide and aide_gui is a good solution

Sequential use of aide modules

We discussed the idea that we would want to make aide modular so that each module could be self-contained, and that modules shouldn't reference one another. If we make aide_gui the controller, we can maintain this modularity while still being able to run each module after the preceding module is done.

Here's some pseudocode for how this would look, based off of the current state of aide_gui.py:

def run():
    showPalette()
def showPalette():
    HTMLEvents.beginHandling()
    DesignEvents.beginHandling()
    DrawDocEvents.beginHandling()
def HTMLEventHandler():
    navigate_gui()
    if HTMLEvent is user_input:
        write(user_input, yaml_path)
        DesignEvent.execute()
def DesignEventHandler():
    # NOTE: The location of the design repository and its associated files can be stored in the YAML.
    aide_design(yaml_path)
    DrawDocEvent.execute()
def DrawDocEventHandler():
    aide_draw(yaml_path)
    aide_document(yaml_path)

Communicating with the user

Combining aide and aide_gui would make it much easier to convey when each part of the design process is finished by displaying a different page when each module is being run. Additionally, it would make for an overall better experience for the user when they can see what aide is doing at the moment.

Team restructuring for Fall 2018

I've been thinking about how we can improve upon the current structure of the AIDE subteam and where we're headed in the coming academic year. Here are some ideas I have:

More Github, less GDrive

Upon cleaning out the AIDE GDrive folder (which was pretty messy), I realized that the vast majority of our organizational tools can be ported over to the functionality in Github: issues for to-do's, discussion boards for status updates, etc. I think the GDrive folder should only be used for:

  • Files that don't work well with Github repos (e.g. our presentations, drawings/diagrams)
  • Documents that need to be updated and written to by multiple people at the same time

Everything else should go in AIDE repositories, wikis, or AIDE teams' discussion boards. The benefit of this is that it reduces the amount of maintaining we'll have to do, and it will make AIDE team members more comfortable with using Github - something they'll inevitably have to do in their future careers.

Less specialization of coding teams

AIDE members almost always fall into one of two skill sets: CAD-ing and coding. There's definitely no expectation that team members should switch between those two skill sets. As such, AIDE Template should remain as a discrete (sub-)sub-team within AIDE.

But instead of having sub-teams that each specialize in a single AIDE module, I believe we should have general coding teams that can work on any module necessary. This could take the form of 2-3 coding teams (called AIDE Code A/B/C, or perhaps something that doesn't imply a hierarchy ๐Ÿ˜), such that each team's members are working together for the entire semester. These coding teams wouldn't be tied down to a single AIDE module, and instead would have the freedom to switch between working on different modules.

Benefits

  • Team members will have a better, broader understanding of the entire AIDE tool and different software development tools
  • Exposing more people to the same problem can bring in new ideas
  • Avoid burnout on difficult tasks

Drawbacks

  • It will take much longer for a new team member to jump in
  • Switching modules/problems will take extra time and energy

Better coherence as a team

To resolve some of the above drawbacks, we should actively build coherence of AIDE as a centralized subteam, rather than a bunch of sub-sub-teams. When I first joined AIDE, I didn't quite understand why I was doing what I was doing, and as a result, our work was confused and we sometimes had to backtrack. The first priority for AIDE in a given semester is that each team member understands:

  1. What the AIDE tool is intended to accomplish for the end user
  2. How each AIDE module works and interacts within the entire AIDE tool
  3. How to write good code that is easy to read and understand

When new members join, they should be required to complete both the AguaClara Tutorial and the AIDE Tutorial. There's a lot of knowledge in the AguaClara Tutorial that would be useful for AIDE members, especially with using Git and Github. I'm working on improving both tutorials to ease new members into the AIDE workflow.

It's also important that the subteam's leadership is actively interacting with team members so that they know someone to turn to when they run into problems. For this, we could hold monthly team meetings - similar to the current leadership meetings, but with more involvement (as AIDE is so large, I don't expect everyone to be able to attend, but at least one person from each team should).

To facilitate teams switching between different modules/problems, we should have "pivots": meetings in which one team teaches another team about what they've accomplished with their module, and provides tips on how to continue. These pivots will continually build on the team members' understanding of AIDE.

Semester-long planning

There are about 15 weeks from when new AguaClara team members join to the final presentations - we'll have to plan wisely. A possible schedule:

  • 1-2 weeks: Learning about how AIDE works, old members teach new members
  • 5-6 weeks: Adding features, tackling problems, pivots, team meetings
  • 1 week: "Frozen code", during which no new features are added. Ensure that code is well written and documented, culminating in code reviews
  • Mid-semester Symposium
  • 5-6 weeks: Adding features, tackling problems, pivots, team meetings
  • 1 week: "Frozen code"
  • Final Presentations

LFOM Smart Component

This is a reminder to comment and finish the LFOM code and validate the full lifecycle.

Review variable naming once MVP is complete

At the end of the semester, we should review and unify all the variable naming schemes used throughout the MVP. For example, some places abbreviations are written all uppercase, like JSON, but others they are written Json. In our code, this would look like hlBOD vs HLbod.

Python Code Design (Fi)

Create all functions needed for HDF
Verify that siphon design fixes premature leaking observed at Gracias.
Set expert inputs to produce design that is very similar to Gracias design

The OStaRS filter colab sheet is ready for conversion to FeatureScript

Create Mock Scrum

Create a scrum tutorial or mock scrum to help new students understand how they should work

Document LFOM

Automate the creation of design specifications for the LFOM

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.