Git Product home page Git Product logo

aguaclara.github.io's Introduction

aguaclara

Pypi Version Documentation Build Status Code Coverage

aguaclara is a Python package developed by AguaClara Cornell and AguaClara Reach for designing and performing research on AguaClara water treatment plants. The package has several main functionalities:

  • DESIGN of AguaClara water treatment plant components
  • MODELING of physical, chemical, and hydraulic processes in water treatment
  • PLANNING of experimental setup for water treatment research
  • ANALYSIS of data collected by ProCoDA (process control and data acquisition tool)

Installing

The aguaclara package can be installed from Pypi by running the following command in the command line:

pip install aguaclara

To upgrade an existing installation, run

pip install aguaclara --upgrade

Using aguaclara

aguaclara's main functionalities come from several sub-packages.

  1. Core: fundamental physical, chemical, and hydraulic functions and values
  2. Design: modules for designing components of an AguaClara water treatment plant
  3. Research: modules for process modeling, experimental design, and data analysis in AguaClara research

To use aguaclara's registry of scientific units (based on the Pint package), use from aguaclara.core.units import u. Any other function or value in a sub-package can be accessed by importing the package itself:

Example Usage: Design

import aguaclara as ac
from aguaclara.core.units import u

# Design a water treatment plant
plant = ac.Plant(
    q = 40 * u.L / u.s,
    cdc = ac.CDC(coag_type = 'pacl'),
    floc = ac.Flocculator(hl = 40 * u.cm),
    sed = ac.Sedimentor(temp = 20 * u.degC),
    filter = ac.Filter(q = 20 * u.L / u.s)
)

Example Usage: Core

# continued from Example Usage: Design

# Model physical, chemical, and hydraulic properties 
cdc = plant.cdc
coag_tube_reynolds_number = ac.re_pipe(
    FlowRate = cdc.coag_q_max,
    Diam = cdc.coag_tube_id,
    Nu = cdc.coag_nu(cdc.coag_stock_conc, cdc.coag_type)
)

Example Usage: Research

import aguaclara as ac
from aguaclara.core.units import u
import matplotlib.pyplot as plt

# Plan a research experiment
reactor = ac.Variable_C_Stock(
    Q_sys = 2 * u.mL / u.s, 
    C_sys = 1.4 * u.mg / u.L, 
    Q_stock = 0.01 * u.mL / u.s
)
C_stock_PACl = reactor.C_stock()

# Visualize and analyze ProCoDA data
ac.iplot_columns(
    path = "https://raw.githubusercontent.com/AguaClara/team_resources/master/Data/datalog%206-14-2018.xls", 
    columns = [3, 4], 
    x_axis = 0
)
plt.ylabel("Turbidity (NTU)")
plt.xlabel("Time (hr)")
plt.legend(("Influent", "Effluent"))

The package is still undergoing rapid development. As it becomes more stable, a user guide will be written with more detailed tutorials. At the moment, you can find some more examples in specific pages of the API reference.

Contributing

Bug reports, features requests, documentation updates, and any other enhancements are welcome! To suggest a change, make an issue in the aguaclara Github repository.

To contribute to the package as a developer, refer to the Developer Guide.

aguaclara.github.io's People

Contributors

eak24 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aguaclara.github.io's Issues

Creating a new architecture using Angular

So we need to move POST into the Angular framework, and I've been thinking that the best way to do this efficiently before moving forward is to think of our current site on the development branch in terms of Angular's Templates, Components, and Services structure.

These are my current thoughts on what parts we need, and please correct me if I'm understanding each part wrong:

Templates

All visual parts of the site

  • Tab structure
  • Internal templates within the above tab structure for tables, graphs, downloads
  • Navigation (plant list)
  • Extra templates for the static pages, like instructions/about

Components

Underlying data structures, linked to a template

  • I was confused about how to structure this, would it be a Plant component that holds all the data, which then gets transformed into the tables/graphs/downloads? Or a plant-table-component, plant-graph-component, etc? What are your thoughts?
  • A plant list component linked to the navigation sidebar

Services

Handles all non-trivial operations, data fetching, input validation, etc.

  • One to handle fetching data from the databases, and data transformation if we decide to interpret/filter
  • One that enables switching between plants and properly displaying the correct data
  • One that serves downloads

Notes

This is just a general idea of what might work, I'm still getting familiar with Angular so I don't know if there's a more efficient or sustainable way of handling this.

  • Where do we call the d3.js visualization library? Would that be a separate file or something handled by a Service?

Give users the ability to chose a timespan for the desired plant data

Hi @ethan92429!

Just wanted to give you an update of my overall task at the moment. I'm trying to give users the ability to choose what data is displayed on the website by either choosing day, week, month, or year which will result in different queries when the function updatePlantData is called. To do this, I've added some buttons but I'm having trouble getting the functionality to work properly.

I was wondering if you could explain this chunk of code to me. It's in index.html. Right now, my main issue is that I need to pass codeList as a parameter into updatePlantData, but I can't figure out how to do that since it's defined in the block of code I pasted below. I'm thinking either that is the issue, or that you can't just call updatePlantData and expect it to update. Any help would be appreciated, thanks!

*Since this is a pretty complex problem, I'd be open for a call to work through it if you have time! I'll be in the AguaClara lab starting at 4:30 today

$(document).ready(function() {
        submitRadioValues = function(){
          plantCode = $('input[name="plantSelection"]:checked').val();
          save("plantName", plantCode);
          $("#allPlantsDropDown").html("");
          connectSyncButton();
          codeList = [plantCode];
          initViz(codeList);
          $("#datosHeader").html("Datos Sobre la Planta de "+plantCode);
          $("#syncButtonDiv").html("<a id='sync-viz' class='waves-effect waves-light btn'>Sincronizar</a>");
          connectSyncButton();
          addSpinner('#spinnerDestination');
          updatePlantData(visualize, codeList, "month");
          $("#selectPrompt").html("");
        }


        if (getPlantName()==null){
          window.location.replace("/settings.html");

        }else{
          $("#selectPrompt").html("");
          $("#syncButtonDiv").html("<a id='sync-viz' class='waves-effect waves-light btn'>Sincronizar</a>");
          var codeList = [getPlantName()]; //list of currently chosen plants (by code)
          connectSyncButton();
          initViz(codeList);
          addSpinner('#spinnerDestination');
          updatePlantData(visualize, codeList, "month");
        }
      });

Generate monthly water quality reports to download

Oscar, the water system manager in Moroceli would like to see weekly or bi weekly averages to generate a report. The first step here is to create a mockup report to show to Oscar and W+J so they can point out what's necessary and what's not. Then, we can automate the report creation by having the static website hit a Google scripts execution api that fetches and composes data from Fusion Tables, and then injects the data into a google doc that is then brought back to the original caller for download. That doc should also be saved in the drive for future requests of the same time period.

An alternative method would be to automate (using Google Scripts) the sending of such a doc to the Water Board member's personal emails every month or so.

@jonschristensen @wngrimshaw could either of you post here a small sketch/example of the type of report you are wanting us to generate? Just something to give us the gist of what Oscar wants.

Visualizations table plant name not updating

I've only confirmed this issue in the browser version, although I think it exists in the app as well. The table page header always displays "Datos Sobre la Planta de Morocelí", even when viewing the data from Otoro - the plant name just needs to be updated according to the plant being viewed.

Optimize for mobile

On mobile, Chrome continually displays messages saying that it is necessary to download chrome to view the post website. It is possible to continually close the new tab that chrome opens and it is possible to view the data.

Style guide

Create a style guide for the website. i.e. decide on colors, design of navigation bar, uniform fonts (change from Roboto?), distinct font sizes and weights for specific things.

make x-axis on graph in Spanish.

Hey @nhbala can you change the x axis (which has the months) to be in Spanish? Or better yet, add a local variable that is "language" that you use to 'select' Spanish - don't actually have the user select it as that would take too much time right now, but use that value to ask Plotly for the x axis in Spanish if that is possible. See here on how unfortunately Plotly doesn't support localization natively, so we'll have to manually add these tick marks.

view backwash data

The backwash data would be useful as we try to figure out how to improve our filters. We need to add the head loss as a line graph, and backwash events as vertical lines on the graph.

view messages sent by operators

The operators are sending messages and it isn't possible to view them using the POST website. Messages should be visible as a separate tab in between "Tablas" and "Descargas" and could present as a rolling list of incoming messages that dynamically update as you scroll down to load more messages. You'll have to find a way to do this with the JAM stack, which might be a bit tricky.

Redesign time series layout

Bring browser version out of the mobile demo phase and use all the white space!

  1. Stretch out the display
  2. Include Honduran average and EPA standard lines
  3. Redo organization of current user interaction

Fix graphs in IE and Firefox

@jacuerp I know the graph for Firefox is not showing up correctly... Think it has something to do with NaN parsing. This makes me think that this website might not work in ie as well.

Create Two Graphs

Two graphs need to be displayed at the same time, due to different y axis

Make app install instruction page

Make a page that has all the instructions to download both the Collect and the Visualizations app. This should also have a note on updating the apps. This needs to get translated into Spanish. Start by looking here for a start on the ODK Collect installation instructions. I am not sure how to deal with getting the credentials for submitting data, as we do not want to publish those publicly.

Add links to the monthly water reports

We have to make sure that the reports are accessible on the website in an organized fashion. The simplest solution is to have a link to the drive folder that contains all the reports. However, from a user perspective that is clunky and hard to search through (doesn't scale well when we have 100+ reports). One step up would be to have the reports organized by plant that each plant has a different folder within the main 'reports' folder. Furthermore we could organize by year and month. In conclusion, to satisfy this issue I propose we create a link to a reports folder that we will maintain with increasing hierarchical order (using logic in the scripts app we use to generate the reports) and we need another link to the latest report for each plant... be sure to think about how to automatically link these to the latest report... I recommend having a 'staging area' drive folder for each one of the plants that holds the latest reports for each respective plant, and deleting and adding documents using scripts logic. @jacuerp how does this plan sound as a balance between developer lift and user ease?

Displaying both 1 L/s data and main plant data on same graph

In order to display both 1 L/s data and main plant data on the same graph, plot.js needs to be changed. In particular, the makeXScale function would need to be changed so that it scales the axis based on the minimum and maximum values in both timeCollected and timeCollected_L. Checkboxes need to be made that allow the user to select what 1 L/s data to display. The drawLines function needs to return x and y based on timeCollected and/or timeCollected_L, depending on which checkboxes are filled.

@ethan92429 any opinion on whether 1 L/s data and main plant data should be displayable on the same graph or on two separate graphs?

Find and implement MVC framework

We have been missing an MVC framework. Our implementation falls far behind in that we rely on directly interacting with the localStorage, we have disorganized file structure, and we couldn't even add users if we wanted to. In my searches I've found ToDoMVC a curated site that helps compare the various MVC options with an identical app built in each popular MVC framework. My conclusion (as an unexperienced front-end coder) is that we should be using Angular 2, an MVC framework supported by Google, and one of the most popular MVCs on the web. With this, we could easily implement users, components, custom data views, etc. @fawnwong which MVC do you prefer? Do you have experience with writing for an MVC?

Make website jekyll friendly

This includes writing a config.xml that defines any global variables/anything else, making effective use of the includes and layouts functionality of Jekyll.

Noticed that we're getting bad data!

@ethan92429
It seems some of the values we're getting are not being caught by the filters we put in place to get rid of extreme values more than three standard deviations away because extremely large values that are removed still contribute to the sum calculations needed to calculate the standard deviations. Basically, very large turbidity values make standard deviations extremely big so our filter is insensitive to other big turbidity values.

In particular, a raw water turbidity of 40984516.34 was recorded for Cuatro Communidades, but there also negative turbidity values as well which is pretty concerning

To try and resolve this issue, we tried running the filter extremes function multiple times and that did result in better looking graphs that filtered both the extremely high values as well as the high values. However, we opted not to keep the multiple filters because doing so would make us insensitive to poor data coming in. In other words, we wouldn't be able to tell that extremely large data was being inputted by the operators if we kept filtering them out. We'd like to discuss at the next meeting what we should do about this issue and decide if we want a "management" graph, as well as a graph for the public to view.

Visualizations sync issues with Internet Explorer

Antonio noticed this morning that Internet Explorer was having trouble syncing data to display the graph and table. The sync button responded to a click, but then the circle sits there spinning. We tried in Chrome and had no problems, and I was able to replicate the same issue on my machine.

Visualizations browser table sliding header

In the web browser version of the visualizations, could the header of the table where all of the parameters are listed be made to scroll down as the user scrolls, so they don't lose track of the order?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.