Git Product home page Git Product logo

synesthesia-ecosystem's Introduction

Synesthesia Ecosystem

This is a place to share, discuss, request, track, and provide feedback for community efforts, tools, guides, etc., for Synesthesia.

The enhancement issues are not meant to be implemented here, but they can link to the implementation, discussions, further descriptions, etc. The main idea with the tag is to provide an overview of feature suggestion and ease collaboration for anyone who wants to help or are already implementing the features.

Contributing

The simplest way to contribute is to join the discussion in the existing Discord community in the #tools and other #channels, or in this repo's discussion tab.

Other ways are to

  • ask for, list, describe, expand on, ..., features/resources/ideas in the issues
  • edit this file
  • or add new files for more in-depth information.

The #feature-suggestions channel in Discord may prove an inspiration. Some of these features can be implemented outside Synesthesia, which lets the team focus on core features instead.

Table of Contents

Shaders

Libraries

Guides

Show variables

Version 1.19 included the ability to set controls from the javascript file. This allows visualising values in the controller view.

For the built in Synesthesia variables add these to your script.js and scene.json files:

This can be used to visualise any value, including combined values, e.g. Math.sin(TIME)+Math.tan(OnBeatTime).

YouTube channels

Art style

Glitch Art

Tools

Texture sharing

With these tools you can send video from one application to another.

Spout and Syphon Recorder are more performant 1. For remote sharing, you can use NDI.

Audio sharing

VSCode extensions

  • glsl-canvas

    The extension opens a live WebGL preview of GLSL shaders within VSCode by providing a Show glslCanvas command.

    • glsl canvas requires small adjustments to the file, which will be solved by #3.
  • snippets

Scripts

APIs

  • syn-http

    This is a HTTP endpoint for Synesthesia.

Projects

Games

  • syn-games

    A suite of tools that allow various interactions with Synesthesia with the goal of creating new types of games: Sports, board games, language learning, song/music lessons, etc.

    • This repo will be refactored into smaller packages soon, and the documentation improved.

Running on Linux

It's possible to let Synesthesia run on Linux via Wine. It was tested on Arch Linux (on 2021-01-07) with Synesthesia (versions 1.18.1.42 and 1.19.0.51), Wine (version 7.0rc4-1) and an NVIDIA GeForce GT 730 (and the NVIDIA drivers in version 470.94-2). You can also install the Wine Gecko package of your Linux distribution (e.g. for Arch Linux available as wine-gecko) or just let Wine download Gecko when it prompts you to (seems to be when Synesthesia updates are available). You also likely need to change the PulseAudio stream Synesthesia is reading from to the monitor of the stream you want to route to Synesthesia (you can verify that it's working when both PulseAudio and Synesthesia show your audio signals).

The Wine prefix itself needs vcrun2015 and corefonts and the trick is to remove the libEGL.dll shipped by the installer, as it (for some reason) doesn't get along with Wine or the rest of its environment. All but Spout works fine (though Spout wasn't really debugged extensively).

The following commands produce a Wine prefix which worked in at least one case:

WINEPREFIX=~/your-prefix-path WINEARCH=win32 wineboot -u # creates a new Wine prefix at the specified path
WINEPREFIX=~/your-prefix-path winetricks -q vcrun2015 # installs Visual C++ Redistributable 2015 into the prefix
WINEPREFIX=~/your-prefix-path winetricks -q corefonts # installs Microsoft core fonts into the prefix
WINEPREFIX=~/your-prefix-path wine synesthesia-installer.msi # installs Synesthesia, just use all the defaults
rm ~/your-prefix-path/drive_c/Program Files/Synesthesia/libEGL.dll # removes the libEGL.ddl shipped directly with Synesthesia, which for some reason doesn't work in Wine
WINEPREFIX=~/your-prefix-path wine pfx/drive_c/Program\ Files/Synesthesia/Synesthesia.exe # finally start Synesthesia through Wine

FAQ

Footnotes

Footnotes

  1. more performant: Syphon/Spout happens on the GPU, where as NDI forces the GPU to sync with the CPU

synesthesia-ecosystem's People

Contributors

mattorp avatar vsund avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

synesthesia-ecosystem's Issues

Multiple Spout Inputs

I have multiple spout outputs (Spout Senders) from various different applications.

I'd like to be able to select from different spout inputs similar to how we are able to select from multiple camera inputs.

This would behave in the same way Synesthesia currently does in selecting a camera source in having only one active.

In Scene 1, I might choose Spout Sender 1, and in Scene 2 I might want Spout Sender 2, currently you'd have to go into the menu to be able to do this.

Time travel in Synesthesia

Similar to how state management tools in React, this would provide the ability to scrub back and forth in shaders while debugging, but also for use when performing.

This can be combined with looping functionality, to automatically go back to certain points at intervals.

Convert visuals to sound

Going full circle.

By letting visualizers like Synesthesia produce sound it can function as a synthesizer.

Social preview

Upload an image to customize your repository’s social media preview.

Images should be at least 640×320px (1280×640px for best display).

Does anyone have the perfect image for this?

Importing Syn Scene, and ISF scenes

This has been an issue that has been very aggravating to deal with. When trying to load scenes from files Syn only populates Desktop and the user directory. None of the alternate drives or Networked drives populate here, I am having to take them move/split the entire directory that I have organized on specific drives to my Main Drive that is dedicated to programs only not for that.

Recolor options

brandon_stargrave:

Deeper on the fly recolor options via the limit colors meta control would be nice. If it were a customizable color ramp where you can add stops and slide them for weighting, could go a lot deeper than the High - Low.

https://discord.com/channels/729824270913503374/752022203205287936/756238811536556102

Meebs:

For colors, yeah a Ramp thing like TouchDesigner has would be pretty sweet. I'm not sure how they implemented that and kept it fast. But more recoloring options in meta controls (once meta controls become extensible and selectable) seems totally doable.
Having to create a specific GUI element just for the ramp and have that procedurally generate GLSL code to do the recoloring sounds useful but a little intense from a dev perspective, which would push it back to a much later release.
One of the great things about the extensible meta controls (aka post-FX stack) is that each effect could have its own much greater set of controls. Currently the meta controls are limited so that the GUI alone doesn't just get unwieldy.

https://discord.com/channels/729824270913503374/752022203205287936/756321996819923011

Shared repository for shaders

It would allow easy import of functions across projects. Hopefully, it can get newcomers going quickly by easily importing and stitching code together and letting veterans collaborate easier.

Large Media Folder Adding Less Annoying

Please devise a more economical solution for adding folders to Media Folders. Something like Drag-n-Drop, multi folder selections jeez or at least having the add folder button lead you to the "Last known location" would be 100% better than scroll each and every time back to the destination where we select our next categorized media files, just to turn around and have to repeat, repeat, repeat. Spending over an hours worth of time for 20+ categorized media folders we need.

make limit colors automatable

yroc:

even though storing presets generally covers this, making the values for the limit colors midi accessible/automatable could yield some cool results. e.g. for neon vice a couple times i limited a scene to match the neon vice vibe, and it seemed like being able to manually control this without having to make presets could have been cool. not super passionate about this, just an idea that came up....

https://discord.com/channels/729824270913503374/752022203205287936/752748936699314316

Style transfer

TJackal:

What’s the ETA on the style transfer software upgrade? I keep going back to https://youtu.be/I0zG9CYufFQ and it’d be an awesome way to get a leg up on Resolume and TouchDesigner seeing as neither have style transfer capabilities as far as I’m aware.

https://discord.com/channels/729824270913503374/752022203205287936/901068476578414592

Examples of implementation:

Modulation Tools

It would be really nice to have tools to be able to modulate different parameters based on time. (also a Random/S&H is always nice)

It would be really cool if you could assign for instance, an LFO to the direction parameter and over time go from moving forward to backward, etc.

An LFO should be assignable to multiple parameters at once.

Also the ability to create multiple LFOs would be nice, so they can be assigned to different things.

Modulating LFOs with other LFOs is another fun thing to think about.

Might be an "up sell feature", trial gets 1, standard gets 8, pro gets unlimited.

Hot reload

Retain current state when the scene is reloaded.

elapsed time & total/remaining time when playing media

mycsal:

While we’re talking about media a time elapsed/time remaining counter would helpful. I have a folder of random clips of different lengths I pass through and it would be helpful to see how much time is remaining on whatever clips I’m playing. Maybe a way to automatically play the next media in a folder instead of it repeating, and a random button to randomly shuffle through all available media.

https://discord.com/channels/729824270913503374/752022203205287936/752739595312955493

add mappable MIDI control functions to enhance Favslots and Presets

Strauberry:

  • ability to "snapshot" all current parameters to a new Scene Preset via MIDI button

  • ability to auto assign newly created Scene Presets to a new favslot (until all 10 slots are full? when all 10 slots are full start at Slot1 and overwrite?)

  • ability to assign a specific Scene Preset to a MIDI mapping

  • ability to create a mapping to step through (next, previous) Scene Presets (matching the existing previous/next mapping functionality available within the Playlist tab)

having this sort of functionality would allow for enhanced improvisational live performance applications, providing the ability to instantly save and recall Scene Presets without needing to be physically at the host machine. currently the only way to perform these interactions is via mouse and UI.

https://discord.com/channels/729824270913503374/752022203205287936/829833843980107857

Share scene across multiple tools

Different tools, different pros/cons. Being able to use several at the same time could help with development.

Suggested implementation: Write to a file as the scene updates, to allow any tool to update accordingly.

I'll refactor this which should solve a large part of this.

Benchmarking

For deciding on which system/component to buy it would be nice to have access to comprehensive results of different scenes + controls settings. Also for developers to test the performance across systems.

To handle continuous testing, a script to pull, perform and push any test requests not yet completed for a given setup would be helpful. Anyone willing to lend compute power could clone the repo and run the tests automatically at night.

Integrate with GIFs

Possibly as a browser extension or listening to updates in the downloads folder to convert into mp4 and send to the media folder + reload folder in syn.

Customisable UI

Examples:

  • Drag and Drop controls,
  • Hide controls
  • Custom CSS
  • Show live preview of the control's effects instead of, or under, the control
  • ...

Add TOC hook

The Table of Contents should update automatically pre commit.

MIDI out

siegMattel:

whenever I switch scenes and then turn a dial on my midi, it goes totally whack cause the default values don't get sent to the midi to update. expanding on my suggestion of midi out a bit. say I have a midi knob mapped to slider A in scene 1, and I currently have that value set to 0%, controlled by my knob which is also at 0%. I want to switch to scene 2, where I have slider B mapped to the same knob. scene 2's default puts slider B at 100%. When I switch to scene 2, slider B will go to 100%, but the knob on my midi controller will still be at 0%, because it didn't receive the midi out signal from synesthesia updating its value. as soon as I touch that knob, slider B will jump to 0%, and I'll have to quickly turn it back to 100% before controlling it. It's a weird experience and sliders that control things like geometry morphing or speed, it can make a temporarily jarring experience. With MIDI out, any changes to the value of a scene control that is mapped to a midi control will update that midi control if possible (faders can't be magically updated obviously, but things like knobs that just have a virtual position can be). So changing presets, changing scenes, or moving the control within Synesthesia would all update the value of the midi control as well to make a more seamless VJing experience.

https://discord.com/channels/729824270913503374/752022203205287936/752588414360223826

Meebs:

@siegmattel on the topic of Midi Out one of the holdups is that I don't know all the ins and outs of using it because honestly I've almost always just used dumb midi devices. Does the device itself usually do the "catch" behavior, or is that supposed to be managed by the application? How many devices (percentage-wise) support Midi Out? Is there one way of talking to all Midi Out enabled devices or do different manufacturers have different ways they prefer to work? There are maybe 5 or 6 more big questions that I'd probably need to answer before I'd want to take on Midi out.

https://discord.com/channels/729824270913503374/752022203205287936/758451544793677884

siegMattel:

@Meebs Oh man. I'm honestly not sure, you'd probably need to get with someone who has a much better understanding of midi signals and how they can be managed by different applications. I would assume the application does the sending and the device just receives it. I would assume any device that has digital controls can support it, though I'm not positive. I wish I knew more and could be more helpful

https://discord.com/channels/729824270913503374/752022203205287936/758478292835237890

Meebs:

I did just get two new "smart" Midi devices, so as I play w them this fall I'll get a little more familiar with em. I also used one w/ Resolume recently and yeah, it was great to see what clip was selected on each layer and be able to trust my Midi device's light readout

https://discord.com/channels/729824270913503374/752022203205287936/758478694326206514

Separate midi configs for multiple instances

make it easy to store midi configs for multiple instances running in parallel and have each instance have its own default. as it is now on mac, a backup needs to be imported to each 2nd instance. would be good to decouple saving the midi configs from the backup process as well, to make file management easier

by: yroc

https://discord.com/channels/729824270913503374/752022203205287936/754907077448564796

Note: Team Syn is working on official multi shader support: https://discord.com/channels/729824270913503374/752022203205287936/755627072176783481

Workarounds

(Windows) Script to launch two instances of Synesthesia with two sets of MIDI bindings by @headstash:
https://gist.github.com/headstash/242a61afeefc5c477c70cf4a31f41e27

  • create similar script for macOS: #17

This should also be doable with https://github.com/mattorp/syn-server if midi capture is implemented, https://github.com/mattorp/syn-server/issues/6

Then each instance can listen to individual virtual devices, that have custom mappings to them. Compared to pointing to separate binding files, this allows dynamic updating after the instances have launched. Even if Syn will support multiple shaders, custom routing to instances should allow some creative solutions.

Create guide for vscode.dev

With the recent launch of vscode.dev anyone can quickly begin coding, without having to install anything.

But a simple guide for this might be helpful, targeting someone who has never written code before, if possible.

v-sync control

Abstractamcr:

couple things and maybe they have been suggested before but i would love to have v-sync on the controls and be able to assign a midi button to turn it on and off. Also for a loop toggle would be helpful.

UFFFD:

I don't think that has been suggested actually. what's the use case? just using it enable faster visual effects?

Abstractamcr:

@uFFFD i find it really handy switching betteen vsync on and it off. some shadders behave so differently when vsync is off than when its on. Since i have discovered this (thanks to a question from @Jungle Boswick ) I have been finding myself toggling over to settings to click it on and off fairly often, which takes my eyes off the show a bit longer than is ideal. find myself doing it about 4-5 times a show over the course of 3 hours. would be super handy to have that be a button

https://discord.com/channels/729824270913503374/752022203205287936/882320026550562837

Custom keybindings

Z3NPI:

I'm currently using Touch Panel on my iPad to control scenes and effects in OBS and interact with twitch, I have an old DJ controller hooked up with some simple mappings to control syn but I'd like to also have that on my TP -- is there any plans to add keyboard shortcut mappings to syn (which is what TP would use) or has anyone played around with using a soft-synth? Or, even better yet - direct integration with Touch Portal in syn?

https://discord.com/channels/729824270913503374/752022203205287936/776171800823463966

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.