Git Product home page Git Product logo

stlite's Introduction

Stlite: In-browser Streamlit

Serverless Streamlit Running Entirely in Your Browser

Test, Build, and Publish Build and Deploy GitHub Pages

npm (scoped) npm (@stlite/desktop)

Streamlit is a Python web app framework for the fast development of data apps. This project is to make it run completely on web browsers with the power of Pyodide, WebAssembly (Wasm)-ported Python.

Try it out online (Stlite Sharing)

Visit Stlite Sharing.

Create a desktop app (@stlite/desktop)

See @stlite/desktop.

Use Stlite on your web page (@stlite/mountable)

You can use Stlite on your web page loading the script and CSS files via <script> and <link> tags as below. Here is a sample HTML file.

<!doctype html>
<html>
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1, shrink-to-fit=no"
    />
    <title>stlite app</title>
    <link
      rel="stylesheet"
      href="https://cdn.jsdelivr.net/npm/@stlite/[email protected]/build/stlite.css"
    />
  </head>
  <body>
    <div id="root"></div>
    <script src="https://cdn.jsdelivr.net/npm/@stlite/[email protected]/build/stlite.js"></script>
    <script>
      stlite.mount(
        `
import streamlit as st

name = st.text_input('Your name')
st.write("Hello,", name or "world")
`,
        document.getElementById("root"),
      );
    </script>
  </body>
</html>

In this sample,

  • Stlite library is imported with the first script tag, then the global stlite object becomes available.
  • stlite.mount() mounts the Streamlit app on the <div id="root" /> element as specified via the second argument. The app script is passed via the first argument.

⚠️ If you are using backticks ` inside your app script (e.g. if you have included markdown sections with code highlighting) they would close the script block in st.mount(` ... `). To avoid this, you can escape them with with a preceding backslash \.

<script>
  stlite.mount(
    `
import streamlit as st

st.markdown("This is an inline code format: \`code\`")
`,
    document.getElementById("root"),
  );
</script>

More controls

If more controls are needed such as installing dependencies or mounting multiple files, use the following API instead.

stlite.mount(
  {
    requirements: ["matplotlib"], // Packages to install
    entrypoint: "streamlit_app.py", // The target file of the `streamlit run` command
    files: {
      "streamlit_app.py": `
import streamlit as st
import matplotlib.pyplot as plt
import numpy as np

size = st.slider("Sample size", 100, 1000)

arr = np.random.normal(1, 1, size=size)
fig, ax = plt.subplots()
ax.hist(arr, bins=20)

st.pyplot(fig)
`,
    },
    streamlitConfig: {
      // Streamlit configuration
      "client.toolbarMode": "viewer",
    },
  },
  document.getElementById("root"),
);

Various ways to load files (files option)

You can pass an object to the files option to mount files onto the file system, whose keys are file paths, and you can specify the values in various ways as below. See also the File system section for more details.

Passing string or binary data

You can pass the file content as a string or binary data.

This is what we did in the example above.

stlite.mount(
  {
    files: {
      "path/to/text_file.txt": "file content",
      "path/to/binary_file.bin": new Uint8Array([0x00, 0x01, 0x02, 0x03]),
    },
    // ... other options ...
  },
  document.getElementById("root"),
);

Passing an object with a URL

You can use this way to load a file from a URL and mount it to the specified path on the virtual file system.

Either an absolute or relative URL is accepted. Consider as the same as the url option of the fetch() function.

stlite.mount(
  {
    files: {
      "path/to/file": {
        url: "https://example.com/path/to/file",
      },
      "path/to/file2": {
        url: "./path/to/file",
      },
    },
    // ... other options ...
  },
  document.getElementById("root"),
);

Passing an object with options (advanced)

Stlite runs on Pyodide, and it has a file system provided by Emscripten. The files specified via the files option are mounted on the file system, and Emscripten's FS.writeFile() function is used internally for it. You can specify the options (opts) for the FS.writeFile(path, data, opts) function as below.

stlite.mount(
  {
    files: {
      "path/to/text_file.txt": {
        data: "file content",
        opts: {
          encoding: "utf8",
        },
      },
      "path/to/file": {
        url: "https://example.com/path/to/file",
        opts: {
          encoding: "utf8",
        },
      },
    },
    // ... other options ...
  },
  document.getElementById("root"),
);

Loading archive files (archives option)

You can load archive files such as zip files, unpack them, and mount the unpacked files to the file system by using the archives option.

The url field of each item accepts either an absolute or relative URL. Consider as the same as the url option of the fetch() function.

The downloaded archive file is unpacked by the pyodide.unpackArchive(buffer, format, options) function. You have to pass the rest of the arguments of the function, format and options as below.

mount(
  {
    archives: [
      {
        url: "./foo.zip",
        // buffer: new Uint8Array([...archive file binary...]), // You can also pass the binary data directly
        format: "zip",
        options: {},
      },
    ],
    // ... other options ...
  },
  document.getElementById("root"),
);

Multipage apps

You can pass the multiple files to the files option as below to construct the multipage app structure, the entry point file and pages/*.py files.

Read the Streamlit official document about the multipage apps.

stlite.mount(
  {
    entrypoint: "πŸ‘‹_Hello.py",
    files: {
      "πŸ‘‹_Hello.py": `
import streamlit as st

st.set_page_config(page_title="Hello")
st.title("Main page")
`,
      "pages/1_⭐️_Page1.py": `
import streamlit as st

st.set_page_config(page_title="Page1")
st.title("Page 1")
`,
      "pages/2_🎈_Page2.py": `
import streamlit as st

st.set_page_config(page_title="Page2")
st.title("Page 2")
`,
    },
  },
  document.getElementById("root"),
);

Customizing the Streamlit configuration (streamlitConfig option)

You can pass the Streamlit configuration options to the streamlitConfig field as key-value pairs as below. Unlike the original Streamlit configuration, the options are passed as a flat object with the keys separated by dots.

stlite.mount(
  {
    streamlitConfig: {
      "theme.base": "dark",
      "theme.primaryColor": "#00b4d8",
      "theme.backgroundColor": "#03045e",
      "theme.secondaryBackgroundColor": "#0077b6",
      "theme.textColor": "#caf0f8",
      "client.toolbarMode": "viewer",
      "client.showErrorDetails": false,
    },
    // ... other options ...
  },
  document.getElementById("root"),
);

Different Stlite versions

In the example above, the Stlite script is loaded via the <script> tag with the versioned URL. You can use another version by changing the version number in the URL.

The following URLs are also available, while our recommendation is to use the versioned one as above because the API may change without backward compatibility in future releases.

The latest release

<script src="https://cdn.jsdelivr.net/npm/@stlite/mountable/build/stlite.js"></script>

You can use the latest version of the published Stlite package with this URL.

The head of the main branch

<script src="https://whitphx.github.io/stlite/lib/mountable/stlite.js"></script>

This URL points to the head of the main branch which is usually ahead of the released packages. However, we strongly recommend NOT to use this URL because this might be broken and there is no guarantee that this resource will be kept available in the future.

Different Pyodide distributions (pyodideUrl option)

Stlite uses Pyodide and loads it from the CDN by default. You can use your own Pyodide distribution by passing the URL to the pyodideUrl option as below. This would be helpful for example when your organization has a restrictive policy for CDN access.

stlite.mount(
  {
    pyodideUrl: "https://<your-pyodide-distribution-url>/pyodide.js",
    // ... other options ...
  },
  document.getElementById("root"),
);

Pyodide provides two distribution types, full and core, and you should serve the full distribution in this case. Stlite loads some packages from the Pyodide distribution such as micropip and they are not included in the core distribution. Even with the full distribution whose size is quite large (+200MB), only the necessary packages are loaded on demand, so the actual size of loaded resources is smaller and you don't have to choose the core distribution worrying about the size. Ref: #1007.

File system

Stlite executes your Python code on Pyodide in the browser, and Pyodide has its own Linux-like file system isolated from the host OS (see Pyodide's or Emscripten's documents about the FS for details). The source code and data files are mounted on the file system through the files and archives options as described above, and the Python code can access them. So, for example, what open("path/to/file") reads or writes is the file on the file system virtually existing in the browser, not a file on the host OS.

The default file system (MEMFS) is ephemeral, so the files saved in the directories are lost when the page is reloaded. The root / and some directories including home are mounted as MEMFS, the ephemeral file system, by default.

File persistence with IndexedDB backend

To persist the files across the app restarts, you can use the IndexedDB-based file system (IDBFS). The files saved in the directories mounted with IDBFS are stored in the browser's IndexedDB, so they are persistent across the app restarts.

In the case of @stlite/mountable, you can mount the IndexedDB-based file system, IDBFS to the specified directories in the virtual file system, by passing the idbfsMountpoints option as below. The mounted file system is persistent across the page reloads and the browser sessions.

stlite.mount(
  {
    idbfsMountpoints: ["/mnt"], // Mount the IndexedDB-based file system to the /mnt directory.
    entrypoint: "streamlit_app.py",
    files: {
      "streamlit_app.py": `
import datetime
import streamlit as st

with open("/mnt/data.txt", "a") as f:
    f.write(f"{datetime.datetime.now()}\\n")

with open("/mnt/data.txt", "r") as f:
    st.code(f.read())
`,
    },
    // ... other options ...
  },
  document.getElementById("root"),
);

HTTP requests

To make HTTP requests, these libraries work on Stlite.

Stlite automatically enables koenvo/pyodide-http's patches to make requests and urllib work, while the networking libraries do not work in general on the Pyodide runtime (Python in browser) as written in this doc and Pyodide provides its standard alternative methods to make HTTP requests, pyodide.http.pyfetch() and pyodide.http.open_url().

Also, urllib3 supports Pyodide since 2.2.0 as this document says.

Limitations

As Stlite runs on the web browser environment (Pyodide runtime), there are things not working well. The known issues follow.

  • st.spinner() does not work with blocking methods like pyodide.http.open_url() because Stlite runs on a single-threaded environment, so st.spinner() can't execute its code to start showing the spinner during the blocking method occupies the only event loop.

    • If you want to show a spinner with a blocking method, add a 0.1-second sleep before the blocking method call, although this will definitely add an empty 0.1-second wait to the execution.
      with st.spinner("Running a blocking method..."):
          await asyncio.sleep(0.1)  # Add this line to wait for the spinner to start showing
          some_blocking_method()
  • st.bokeh_chart() does not work since Pyodide uses Bokeh version 3.x while Streamlit only supports 2.x. The 3.x support for Streamlit is tracked here: streamlit/streamlit#5858

  • time.sleep() is no-op. Use asyncio.sleep() instead. This is a restriction from Pyodide runtime. See pyodide/pyodide#2354. The following section about top-level await may also help to know how to use async functions on Stlite.

  • st.write_stream() should be used with an async generator function rather than a normal generator function. Due to the same reason as st.spinner() above, the normal generator function does not work well in the browser environment, while it still can be passed to st.write_stream(). The following is an example of st.write_stream() with an async generator function.

    async def stream():
        for i in range(10):
            yield i
            await asyncio.sleep(1)
    
    st.write_stream(stream)
  • There are some small differences in how (less common) data types of DataFrame columns are handled in st.dataframe(), st.data_editor(), st.table(), and Altair-based charts. The reason is that Stlite uses the Parquet format instead of the Arrow IPC format to serialize dataframes (Ref: #601).

  • Packages including binary extensions (e.g. C/Rust/Fortran/etc) that are not built for the Pyodide environment cannot be installed. See https://pyodide.org/en/stable/usage/faq.html#why-can-t-micropip-find-a-pure-python-wheel-for-a-package for the details.

Other problems are tracked at GitHub Issues: https://github.com/whitphx/stlite/issues If you find a new problem, please report it.

Top-level await

TL;DR: Use top-level await instead of asyncio.run() on stlite.

Unlike the original Streamlit, Stlite supports top-level await due to the differences in their execution models. Streamlit runs in a standard Python environment, allowing the use of asyncio.run() when an async function needs to be executed within a script. In contrast, stlite runs in a web browser, operating in an environment where the only event loop is always in a running state. This makes it impossible to use asyncio.run() within a script, necessitating the support for top-level await.

Top-level await can be useful in various situations.

Example 1: asyncio.sleep()

One of the most common use cases is asyncio.sleep(). As mentioned in the previous section, time.sleep() is no-op on Stlite because its blocking nature is not compatible with the single-threaded event loop in the web browser environment. Instead, asyncio.sleep(), which is non-blocking, can be used to pause the execution of a script for a specified amount of time.

You can use top-level await either for asyncio.sleep() directly or for an async function that contains asyncio.sleep() like the following:

import asyncio
import streamlit as st

st.write("Hello, world!")
await asyncio.sleep(3)
st.write("Goodbye, world!")
import asyncio
import streamlit as st

async def main():
    st.write("Hello, world!")
    await asyncio.sleep(3)
    st.write("Goodbye, world!")

await main()

Example 2: pyodide.http.pyfetch()

Another common use case is accessing external resources. Pyodide provides a Python wrapper of browser's Fetch API, pyodide.http.pyfetch() for making HTTP requests. Since this method is async, top-level await is sometimes used to handle the response.

Here's an example:

import pyodide.http

url = "your_url_here"
response = await pyodide.http.pyfetch(url)
data_in_bytes = await response.bytes()

Resources

Samples

⚑️Serverless Image Processing App

Image processing with OpenCV works on the client side.

See the tutorial video

Serverless Streamlit + OpenCV Python Web App Tutorial, crafted by 1littlecoder.

Serverless Streamlit + OpenCV Python Web App Tutorial

Sponsors

Streamlit (Snowflake)

Databutton

They are sponsoring me on GitHub Sponsors!

Hal9

They are sponsoring me on GitHub Sponsors!

Support the project

ko-fi

Buy Me A Coffee

GitHub Sponsors

Contact the author: Twitter / LinkedIn.

stlite's People

Contributors

andeplane avatar blackary avatar chrieke avatar dependabot[bot] avatar fridokus avatar lukasmasuch avatar robna avatar whitphx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

stlite's Issues

NPM package

  • Developers can use stlite as a NPM package
  • Script tag-based import can be used from CDN source from NPM

St.download

Does st.download works with stlite? I added a download button to my streamlit app using st.lite but it doesn't work.

Desktop/smartphone app bundler

  • Universal: PWA (already)
  • Desktop: Electron/Tauri
    • This may contribute to streamlit/streamlit#1370
    • Electron looks better
      • It has Web Workers support
      • It has browser and NodeJS runtime
      • Tauri seems to be a "web-based UI framework on Rust" on which the web technology is for UI and Rust for the main process.
        • It seems that Tauri does not have a runtime where Pyodide runs.
    • Package name: @stlite/desktop
  • Mobile: Capacitor

Support st.bokeh_chart

It raises an error such as

6.8fc22a8b.chunk.js:1 TypeError: Cannot read properties of undefined (reading 'embed')
    at BokehChart.tsx:109:13
    at BokehChart.tsx:123:5
    at Ls (6.8fc22a8b.chunk.js:1:1608317)
    at t.unstable_runWithPriority (6.8fc22a8b.chunk.js:1:1626130)
    at qi (6.8fc22a8b.chunk.js:1:1548488)
    at Ns (6.8fc22a8b.chunk.js:1:1607778)
    at 6.8fc22a8b.chunk.js:1:1607689
    at B (6.8fc22a8b.chunk.js:1:1625189)
    at E.port1.onmessage (6.8fc22a8b.chunk.js:1:1623911)

Page not found: You have requested page /stlite, but no corresponding file was found in the app's pages/ directory. Running the app's main page.

  • Access to https://whitphx.github.io/stlite/ (the playground app)
  • Wait for the Streamlit app to be loaded
  • The browser URL history is pushed to https://whitphx.github.io/
    • This is not HTTP redirect. It looks like history API manipulation
  • At the same time, the following error dialog appears
    γ‚Ήγ‚―γƒͺγƒΌγƒ³γ‚·γƒ§γƒƒγƒˆ 2022-06-08 0 06 53
    Page not found
    You have requested page /stlite, but no corresponding file was found in the app's pages/ directory. Running the app's main page.
    

This seems to be derived from upgrading Streamlit to 1.10.0: #40 and its multipage app functionality.

Run stlite on edge servers, e.g. fly.io

Run stlite on Cloudflare workers some PaaS where NodeJS can run such as fly.io or adaptable as a server-side process and serve the page like normal Streamlit.
It may be kind of simpler than stlite on browsers as the necessary part may be patching Tornado to run and handle HTTP req/res on the workers and the upstream Streamlit may run on it without modifications.


UPDATE: Cloudflare Workers are not for such usage as its worker function is expected to run in a short time like 10ms, and even on a paid plan, 30mins.
UPDATE2: Cloudflare Workers doesn't limit the duration: https://developers.cloudflare.com/workers/platform/limits/#duration

Introduce ServiceWorker to intercept normal HTTP req/res

Apart the WebSocket communication, Streamlit also uses many HTTP endpoints for various purposes including serving media files and custom component resources.
To make use of these existing HTTP endpoints and the corresponding client implementations with a minimum amount of changes, it looks like ServiceWorker is a good way, compared to re-implementing these communications on top of WebWorker messaging.

Considerations

  • ServiceWorker works in a "progressive" manner, so its full functionality may not be available at the initial time before downloading completes.
  • ServiceWorker may fail to be installed if the script origin is different from the page origin: w3c/ServiceWorker#940

VSCode Extension

  • Preview in WebView
  • Auto-reload

By making it as a web-compatible extension, we will be able to use it on GitHub Codespaces which realizes the true in-browser development experience.

Support components that use PyArrow

Pandas-related components such as st.table or st.line_chart are using PyArrow to pass the data from the server to the frontend, but PyArrow is not working.
PyArrow includes C-extension that we need to build specifically for Pyodide.

    1. Build PyArrow for Pyodide
    • I tried this but it was not successful, at least in my env; M1 mac + Docker + Pyodide official docker image -> Try it on Linux?
    • Pros
      • Streamlit has already had code using PyArrow, and we do not need to rewrite it.
    1. Find a way not to use PyArrow
    • Pros
      • Maybe easier to implement
      • Arrow binary is not needed to load, which means the runtime size can be smaller.
    • Cons
    • Strategies:
      • a) Using the JS version of Arrow and porting it from JS to Python
        • Pros
          • We can use the existing proto
        • Cons
          • Inefficient: We have to transport the data twice: the first is through Pyodide's proxy to use JS arrow, and the second is through the proto
      • b) Create an alternative data transport layer instead of PyArrow
        • For example, pyodide/pyodide#1168 has been solved by using Pyodide's getBuffer for the needs of ndarray.
        • Pros
          • Efficient: We can transport the data from Python to JS through Pyodide's proxy objects.

Read python script from url

It would be nice to just give the url to an existing streamlit app (python code) hosted on github, as we normally do with Streamlit Cloud.

Great work!

Multiple scripts

Support inputting multiple Python files that are imported from the main script file.


The mountable API would be like this?
This is more general design that covers not only Python scripts but also other types of files.

stlite.mount({
  entrypoint: "streamlit_app.py",
  files: {
    // Python files
    "streamlit_app.py": `
import streamlit as st
...
    `,
    "pages/page1.py": `
# This is the first page.
    `,
    "pages/page2.py": `
# This is the second pae.
    `,
    // Other files, e.g. data files
    "data/foo.csv":  // Text file
`id,name
1,Bob
2,Alice
`,
    "data/secret.dat": new ArrayBuffer(/* Some data */), // Binary file
    "models/foo.h5": fetch("...")  // Support async API?
    "models/compressed.h5": {
      url: "....zip",
      unpack: true. // Unpack the archive: https://pyodide.org/en/stable/usage/faq.html#how-can-i-load-external-files-in-pyodide
    }
  }
})

Support st.spinner

It uses threading that is not supported in the Pyodide environment and raises an exception, RuntimeError: can't start new thread.

RuntimeError: can't start new thread
Traceback:
File "/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 555, in _run_script
    exec(code, module.__dict__)
File "/home/pyodide/streamlit_app.py", line 10, in <module>
    with st.spinner():
File "/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
File "/lib/python3.10/site-packages/streamlit/__init__.py", line 441, in spinner
    _add_script_run_ctx(_threading.Timer(DELAY_SECS, set_message)).start()
File "/lib/python3.10/threading.py", line 928, in start
    _start_new_thread(self._bootstrap, ())

Note: time.sleep() is no-op on Pyodide, so awaiting would be a bit tricky.

Reconnect after hot-reload

During development with yarn start in packages/playground, when new changes are saved and the page is hot-reloaded, stlite-kernel is reloaded but Streamlit frontend stays disconnected.
It should reconnect when the kernel restarts.

Loop and/or timer may not work

Reported at https://discuss.streamlit.io/t/new-library-stlite-a-port-of-streamlit-to-wasm-powered-by-pyodide/25556/3

The source code copied:

import streamlit as st
import time

st.set_page_config(
    page_title='Pomodoro',
    layout='centered',
    page_icon='πŸ…'
)

def count_down(ts):
    with st.empty():
        while True:
            mins, secs = divmod(ts, 60)
            time_now = '{:02d}:{:02d}'.format(mins, secs)
            st.header(f"{time_now}")
            time.sleep(1)
            ts -= 1
            if ts < 0:
                break
    st.write("Time Up!")
    st.balloons()

def main():
    st.title("Pomodoro")
    time_minutes = st.number_input('Enter the time in minutes ', min_value=0.1, value=25.0)
    time_in_seconds = time_minutes * 60
    if st.button("START"):
            count_down(int(time_in_seconds))

if __name__ == '__main__':
    main()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.