Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are there callback events for waveform builder? #399

Open
rowild opened this issue Jun 21, 2021 · 2 comments
Open

Are there callback events for waveform builder? #399

rowild opened this issue Jun 21, 2021 · 2 comments

Comments

@rowild
Copy link
Contributor

rowild commented Jun 21, 2021

Displaying long waveforms of long audio files takes time. It would be helpful to show a loading graphic or text while this is happening. To do that, it would be great to have some feedback from the wave building process, sth like '[???].on('start', () => {}), [???].on('end', () => {})`.

The WaveformBuilder uses the waveform-data package, if I am not mistaken, to generate the waveform. I did not recognise any such events.

Is this at all possible? Or is this a feature that would have to be implemented?

On a related note:
The processing using a json (-z 8) via dataUri of a waveform of an audio file that is about 26 minutes in duration, takes pretty long, about as long as via the webAudio API. This is particularly perceptible when switching between zoom levels. It appears the waveform is always generated anew, no matter if zoomLevel 256 has already be rendered once or not.

May I ask: Is that expected behaviour for long files?

I was expecting that waveforms that were already rendered once, would be cached somehow (virtual DOM?) to then be quickly renderable again once that zoomLevel is required again. But that seems to be wrong thinking. (I was influenced by the Liszt example, which is rather fast: http://audiovideo.jeffreyeverhart.com/audio-annotation/peaks.html)

@chrisn
Copy link
Member

chrisn commented Jun 22, 2021

Displaying long waveforms of long audio files takes time. It would be great to should a loading graphic or text while this is happening. To do that, it would be great to have some feedback from the wave building process, sth like '[???].on('start', () => {}), [???].on('end', () => {})`.

The WaveformBuilder uses the waveform-data package, if I am not mistaken, to generate the waveform. I did not recognise any such events.

Is this at all possible? Or is this a feature that would have to be implemented?

Unfortunately this isn't possible, due to limitations in the Web Audio API (decodeAudioData doesn't provide progress feedback, among other issues). The only way around this would be change the waveform-data package to use a JavaScript or WASM audio decoder - or the new WebCodecs API when it becomes widely available.

The processing using a json (-z 8) via dataUri of a waveform of an audio file that is about 26 minutes in duration, takes pretty long, about as long as via the webAudio API. This is particularly perceptible when switching between zoom levels. It appears the waveform is always generated anew, no matter if zoomLevel 256 has already be rendered once or not.

May I ask: Is that expected behaviour for long files?

The default initial zoom level is 256, so I would recommend using the same value when you run audiowaveform, so -z 256. This will avoid the delay on initial render. If you're using a very low zoom level such as -z 8 you will have very large JSON files (e.g., ~140MB for 90 minutes) and changing zoom level will take more time. Do you need the ability to zoom in so much? We chose 256 as a sensible default for our use cases.

(I was expecting that waveforms that were already rendered once, would be cached somehow (virtual DOM?) to the be quickly renderable once that zoomLevel i required again. But that seems to be wrong thinking. (I was influenced by the Liszt example, which is rather fast: http://audiovideo.jeffreyeverhart.com/audio-annotation/peaks.html

Caching the waveform at each zoom level isn't a bad idea.

@chrisn
Copy link
Member

chrisn commented Jun 22, 2021

To clarify: The waveform-data package does two things: generates waveform data from audio, and resamples waveform data to change the zoom level. Generating waveforms from audio has the problem I mentioned above, but resampling is done purely in JavaScript could be changed to provide progress feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants