WebML WG - TPAC 2024 agenda about meetings HOT 13 OPEN

Adding to the proposed F2F topics:

• a refreshed analysis of popular models, operator & data type gaps by @fdwr et al., includes also proposed IDL amendments: webmachinelearning/webnn#375 (comment)

from meetings.

Hi! Over on the Chrome team, we'd love to discuss some of our work on built-in AI APIs with the group. (Currently published: translation and prompting.)

We know this doesn't currently fit in the group's charter, but I think the group previously had a brief discussion about how it might fit in the future. And, we suspect a lot of the right people will be in the room.

We're also happy to join one of the telecons before TPAC if that would be helpful.

from meetings.

Potential topic: Quantization / Dequantization (QDQ)

Including issues such as:

• webmachinelearning/webnn#93
• webmachinelearning/webnn#128
• webmachinelearning/webnn#623
• And specifically past discussions linked from webmachinelearning/webnn#128 (comment)

And topics including:

• Are there key decisions to make, e.g. between operand descriptor encapsulate scale/zp vs. dedicated Q/DQ ops that frameworks introduce (and impls may optimize out) vs. variants of ops specialized for Q data?
• Quantization data types beyond int8 (int4, int2)
• Is support a requirement for v1?

from meetings.

The group discussed F2F agenda on our call today. We reviewed proposals and solicited input on additional F2F topics.

F2F topics proposed in this issue (thanks @inexorabletash @domenic) received support:

• built-in APIs for translation and prompting
• triage pass through open issues at F2F
• quantization and dequantization (QDQ)
• W3C “living standards” topic

@huningxin proposed:

• interop issues across different backends
• next step for implementation, Origin Trial or equivalent and align with framework developer feedback
• incubation for custom ops

@mwyrzykowski proposed:

• MLBuffer and WebGPU interop
• (See also webmachinelearning/webnn#754)

@mmccool proposed:

• incubation for model management w3c/tpac2024-breakouts#15

@anssik proposed:

• platform capability detection (opSupportLimits())
• future-proof device selection abstractions
• "customer-centric session" to hear from people working on frameworks that integrate with WebNN API: ONNX Runtime Web & WebNN EP, demo of Transformers.js v3 with WebNN integration
• ML ethics: WebSHAP tool author @xiaohk from OpenAI / Georgia Tech to present on in-browser explainability libraries and viz tools, ~30 mins for presentation + Q&A, prefers AM PDT timeslot
• Wide review status: close on any remaining TAG review feedback (resource contention considerations ...)
• webmachinelearning/webnn#573 revisited

@domenic, the group is happy to hear more from built-in AI APIs for translation and prompting. We'd like to invite you to attend our TPAC meeting to discuss these proposals with us. Please let us know, including any time preference you may have for 23 Sep.

from meetings.

It might be productive to spend some cycles doing a triage pass through open issues:

• Breaking? Would acting on this issue impact code such as webnn-samples or ONNX Runtime Web?
• Priority? Is this a blocker for milestones such as an "origin trial" of the API?
• Next steps? What action is required, especially for older/broader issues?

This could be all consuming so definitely time-box (overall and per-issue), but real-time discussion could help make progress.

from meetings.

Potential topic: Should we make the spec a "living standard" from the get-go? TPAC might be a good place to identify concerns and walk through this with W3C experts.

from meetings.

@domenic, the group is happy to hear more from built-in AI APIs for translation and prompting. We'd like to invite you to attend our TPAC meeting to discuss these proposals with us. Please let us know, including any time preference you may have for 23 Sep.

Awesome! Coincidentally, we just finished up publishing the rest of our explainers. In total we have:

• WICG/translation-api: translator and language detector APIs
• explainers-by-googlers/writing-assistance-apis: summarizer, writer, and rewriter APIs. We hope to move these to WICG soon: WICG/proposals#163
• explainers-by-googlers/prompt-api: the prompt API, which is extra-experimental and is not yet proposed to moving to WICG as its ultimate destiny is unclear.

We hope to present on all of them together, under the heading of "built-in AI APIs".

I will be available any time on September 23.

from meetings.

@domenic great to hear, we'll come up with a more baked in agenda this month and will slot you in.

Adding to the pile of proposals: for our possible ML ethics / transparency & explainability discussion, adding Visual Blocks visual programming framework. I believe @xenova added a few new nodes to this framework for familiar vision, NLP, other tasks. While not purpose-built to solve ethical issues, this tool kind of serves that purpose too while being educational also. Very demo worthy, see "Example" links at https://huggingface.co/hf-vb for client-only nodes and blog for hybrid server + client pipeline example.

from meetings.

More exciting agenda updates:

• @tqchen is joining us for an update on WebLLM and other projects that build upon the Machine Learning Compilation solution. The group received an introduction from TQ earlier this year. Possible synergies with the proposed incubation for custom ops from @huningxin et al. TQ's availability 12-2 PM PDT (3-5 PM EDT). ~20-30 min slot.

• I'm also pleased to confirm @xiaohk from OpenAI / Georgia Tech will join us to present on in-browser explainability libraries and viz tools. This contributes to our ethics deliverable. Jay's preference AM PDT timeslot.

from meetings.

Regarding AI Model Management, while there is a breakout later in the week I'd like 10-15m in the "main call" to summarize the issues and alternatives and gather input.

from meetings.

The proposed WebML TPAC 2024 agenda topics have been assigned timeslots, please take a look: #25 (comment)

I've assigned the following timeslots to our guest speakers:

📍9:30-10:00 In-browser explainability libraries and viz tools (@xiaohk, OpenAI / Georgia Tech)
📍13:30-14:00 WebLLM + MLC (@tqchen)

For other topics, we have some wiggle room to move things around.

You should have received a meeting invite that links back to this agenda issue. Zoom joining instructions will appear in the meeting invite link around 19 Sep. You can also manually export the invite as needed.

Thank you all for your contributions and proposals. I am looking forward to meeting you all for a day of discussions on these exciting and diverse topics.

from meetings.

I'd like to add the following issue to the agenda:

• webmachinelearning/webnn#668

from meetings.

@reillyeon thanks for the proposal, added to our set of interop issues.

from meetings.