Bugs (CPU, Router view, Audio driver related)

[Eugene]

First, forgive me if I need to use a bugtracker or something.
I am not very aware of how to do this right, but willing to use my efforts to fix them by myself.

So here is the list. From the most important to lowest.

1. Router view.
1.1. All machines show signs of activity while being totally idle. Which eats a lot of CPU as well (60-200%).
1.2. While playing a song, again, high CPU usage (>100%) when looking on router view. Switching to sequencer view for instances reduces usage to 50-70%.

2. Audio driver.
2.1. Sometimes, after opening a song I hear some kind of stuttering of some machines.. Something like a noise in an analogue telephone line. But the CPU usage doesn't increase 70%. Changing the audio driver option vice/versa solves this.
2.2. Also sometimes, when loading Neil I am getting "Unable to open audio device" message. But it was playing well with the same device before closing it!

Now the Specs about my PC:

Audio driver (what I see in device config):
hw:SB Audigy 1 ES [SB0160],0
hw:SB Audigy 1 ES [SB0160],3

CPU: P4 Celeron 2.8 MHZ single core.
Memory: 1GB.

Operating system: Ubuntu 11.10 with pulseaudio removed.
Video card driver: radeon (open source).


I would really like the performance issue to be solved and willing to do this by myself if needed. I have no python knowledge bug very good at programming in general.
I would like to hear on where to look at or where to start looking.

Eugene

[Eugene]

This becomes very critical for me!

I have changed the motherboard to a more Linux-friendly one (regarding graphics), but the issue remains!

I have even disabled all the router graphics in python files and the issue still remains: idle machines alltogether use a lot of CPU! And by using "top" command it shows that this cpu usage is by "python" only (not Xorg for instance), moreover, top's values are higher than those shown inside of the program.

I have tried to ask for the support from paniq by describing the issue on his Armstrong's page:

https://bitbucket.org/paniq/armstrong/issue/19/idle-machines-high-cpu-usage

Also, dont forget, that it's not only me suffering from this:

https://bitbucket.org/bucket_brigade/neil/issue/148/router-performance

I would explain a little more the problem itself: each single machine which is idle but in player play/stop state takes some amount of CPU. Before you press play the first time everything is allright! But once you will press play, even idle machines take CPU! After pressing stop and even not hearing any sound anymore, CPU usage would last forever!
When there are just a few of machines (about 10 in total) it is almost not noticable. All .ccm files provided on this group do not use much more than about 10 machines!

But I personally want to create GOA-Trance music and it generally requires a lot of machines, taking it all from the machine. At least, this is how people were doing.
I also want to learn making music, I am a beginner and also I don't mind learning Linux and contributing to the project (already contributed bypass mode, full psycle plugins support).
There are quite a lot of native psycle modules available and some of them a real GOA-Trance (Taika Kim)! This is the perfect way to learn!
I was also don't mind using psycle under wine, but it becomes also very hard, because older .psy files might not work with newer psycle release and vice versa. So it's messing up with various psycle versions and also wine versions as on newer wine the performance of psycle is much worse somehow..
So I've chosen the way of porting psycle songs which use it's native plugins into Neil. By now by hand and it's quite interesting learning-wise.

This is the first song I've made:

https://soundcloud.com/deepxcode/ht-brain-cry

It is completely sequenced and rendered in Neil using only pscycle plugins. It works just fine as it uses about 10 machines in total.

But when it came to the second song, which uses about 40 machines, it becamse unable to play it! The CPU usage raised to 300% even when only 10 machines out of 40 were playing!
Strange thing, when I removed all of the unused machines for the current pattern, usage dropped to 10%!!
Also, when I only muted all of the machines, usage dropped to 60%.

This is not related to router view only, but any view!

There definitely should be some bug regarding detecting of the idle state of machines. I've started learning all of the possible related files and here is what I have found already (you may also see this in my Armstrong's issue link):

In file libneil/src/libzzub/song.cpp on the line about 1290 there is the code (running each time after command to process audio has been given to a plugin):

if (m.last_work_audio_result) {
  if (scanPeakStereo(&m.work_buffer[0].front(), &m.work_buffer[1].front(), sample_count, 
         m.last_work_max_left, m.last_work_max_right, falloff)) {
// the plugin claims it has generated non-silence, but our scan says otherwise
m.writemode_errors++;
  }
}

 

 I have made a check of adding here a code of writing m.writemode_errors into a file. It has shown that the value gets increased for about a thousand (1000) per second! This is after you would press play. When you would press stop afterwards, this value would get increasing forever!! Unless you would close the program or possibly load another file.

I think this is very significant indication of the bug, but I still haven't figured out, how it generally works: who (which file, line) tells the plugin to play and decided if there is anything to play or no according to the pattern. This seems to be able to improve and do not use more than 10% of CPU if you have 40, 50 or 100 machines in the idle state! I would repeat that this is very critical for me and I am spending all my available time to study the files and gather the information.

I have already contacted the original libzzub developer Anders Ervik (calvin) on #buze IRC channel. People say he is available often but no reply yet.
I have also checked for at least any documentation available over libzzub. And found only very little:

http://web.archive.org/web/20080727012344/http://doc.zeitherrschaft.org/pyzzub/
This is from the former libzzub web site http://web.archive.org/web/20081001113648/http://trac.zeitherrschaft.org/zzub/

There is also a recent documentation for Armstrong available at Buze website:
http://batman.no/buze/developer/armstrong/index.html

But it seems to be very different: SQLite and everything..



When I was writing this, I have got a reply in #buze channel were the original libzzub developer resides and I'm still waiting for him to suggest.
Someone there told me that this issue might be related to very known "denormals" problem.
I have also talked in #psycle channel and it's developer, bohan, confirmed that it might be this and added that they in psycle, take care of this on the host level:

<bohan> in psycle's core, the output of a plugin is passed through a denormal/infinite/nan remover

Do we have things like this Neil?
Is there are LADSPA/DSS module for denormal removing?

And here is a little suggestion fro, #buze channel regarding the song.cpp:

ld0d> b0tm1nd, well, you have to figure out if it's process_stereo() (which should end up calling the plugin itself rather directly) or something else that's taking so much time
<ld0d> b0tm1nd, because process_stereo should be rather directly a call to the plugin itself
<ld0d> b0tm1nd, and if it's not that but the code after it, then the correct way to fix it would be to move the scanPeakStereo() earlier and to clear last_work_audio_result if the output is silent, and fix all the rest of the code to use it
...
<ld0d> but it could be something else, so better analyze process_stereo() carefully first

 
I will remind that you can notice this on any .ccm file availble! Just press play then stop and you will see the CPU usage of machines which should not be!

[DVE]

Hm. My new one has +50 (machines), whereas my laptop has +6 (years old) and everyone's happy.
Do you count effects actually? I remember tritium-3.ccm I posted long time ago had 35 machines if we count effects.
I understand your problem - I noticed SOME cpu usage on idle - but it never became that critical, even thought like I said, my machine is rather old.
Maybe I don't reach that level of machine sophistication:)
regards
wojtek

--
Pentahedron - A polyhedron having five faces. Because there are two pentahedral graphs, there are two convex pentahedra, namely the square pyramid and the triangular prism. The triangular prism is the unique convex pentahedra having unit edge lengths.

(Eric W. Weisstein. "Pentahedron." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/Pentahedron.html)

[Eugene]

Hello!
Thanks for your reply!

At first about the machine. I used 6 years old desktop, yours is laptop, supposed to be faster I believe. Mine is single core P4 celeron 2.2 overclocked to 3.4 ghz.
Maybe you have dual core and it's not that noticable?

Then I haven gotten a lot of feedback from both psycle developer and original libzzub/related developers.
The issue seems to really be about the so called "denormal values" issue.
Here is what it is from http://carlh.net/plugins/denormals.php :

Denormals (or subnormals) are simply very small floating point numbers. When they drop below a threshold size, many CPUs (under many different conditions) exhibit considerable drops in performance when processing them; the drops can be up to a factor of 100 times!

The conclusion from all the conversation I had with other developers is that this might be plugin specific issue who does not take care of denormals issue and which might not be that noticable on more powerful machines than mine.

And yes, further tests pretty much prove the theory that only some plugins (as well as psycle wrapper) are needed to be fixed.
Indeed, the problematic file has 32 machines including effects but all from psycle!
Your files, in particular, tritium-3.ccm and 1-14-removed_samples.ccm have also +-30 machines! I have tested again and noticed, that in tritium-3 mainly 2 machines causes idle state CPU usage: MVerb and Lunar Delay. In 1-14-removed_samples MVerb also behaives this way.

But with psycle almost every single plugin behaives like that which makes it impossible to use.

The solutions might be to re-compile psycle plugins in optimized mode, update psycle plugins wrapper, try to fix MVerb.. Also test this issue deeply with different plugins..
I have quite a lot to try.

And meanwhile, if you could provide any more information, like screenshots of CPU usage of all machines during play/stop, your computer specs and maybe this new file with 50 machines and at least 1 pattern it would be very helpful!! :)


Eugene

[Eugene]

We have some progress already!! :)

Thanks to original libzzub developer (Anders Ervik, calvin, clvn) we have solved a part of the problem!
It was indeed MVerb machine which was causing the problem!
There are rules for making plugins:
http://www.buzzmachines.com/machineprogrammingmistakes.php
Especially the number 2. regarding not to work while processing silence.
MVerb has been left in such a way that it was exactly breaking the rule: working all the time no matter if the input signal was silence or data.
For me each instance of MVerb takes 10% of CPU when it's working.
In tritium-3.ccm there are 3 of them, but at a time mostly only single works. Thats 20% overhead!!
For me it's very significant as the total CPU usage is 70-80% and now 50-70%!

That's a great step forward in the case! There is another plugin, Lunar Delay which behaves just like this, but it's taking about 5% each, but still, if fixed another 5-15% down is great! But it's fixing is harder because if the nature of filter as in case of reverb it's like AAaaaa and you could detect when to stop by the first silence, but in delay, it's like A A a a a , where silence detection method would not work and one needs to calculate the effect's length and after that trigger the stop.. Hopefully someone would implement this soon! :)

Another progress achieved today is with psycle plugins! As been said above, psycle takes care of "denormals" and similar issues on host level, while libzzub/neil does not.
After we applied additional denormals check into psycle wrapper, in stop mode, plugins became calm! Before that in stop mode plugins were using about 70% while more than 100% in play mode, now in stop it dropped to 10% which is actually 0 but play mode is still problematic as various plugins produce work while they are not supposed to.

Hopefully this would been taken care of really soon as well! :)

Though any help is still appreciated!!


Eugene

[Eugene]

For those interested, here is the screenshot of the situation:

http://postimage.org/image/or31zdqhp/

Notice mainly MVerb2 and MVerb3 which take about 20% together but do not produce any sound. Now the problem is completely fixed with them. Some are still left.

[Vytautas Jancauskas]

"As been said above, psycle takes care of "denormals" and similar issues on host level, while libzzub/neil does not."

That is incorrect, it does take care of denormals on host level.

--
"Cheshire-Puss," she began, "would you tell me, please,
which way I ought to go from here?"
"That depends a good deal on where you want to get to," said the Cat.
"I don't care much where--" said Alice.
"Then it doesn't matter which way you go," said the Cat.

[Eugene _]

Could you please specify more detailed?

As for me it seem to have some initial attempts to deal with the problem.

Here is the main audio processing code taken from song.cpp (line about 1233):

    // process audio:
    int flags;
    if (((m.info->flags & zzub_plugin_flag_has_audio_output) != 0) &&
    ((m.info->flags & zzub_plugin_flag_has_audio_input) == 0)) {
      flags = zzub::process_mode_write;
    } else {

      if (result) {
    bool does_input_mixing = (m.info->flags & zzub::plugin_flag_does_input_mixing) != 0;
    bool has_signals =
      buffer_has_signals(&m.work_buffer[0].front(), sample_count) ||
      buffer_has_signals(&m.work_buffer[1].front(), sample_count);
    flags = (does_input_mixing || has_signals) ? zzub::process_mode_read_write : zzub::process_mode_write;
      } else
    flags = zzub::process_mode_write;
    }

    memcpy(&mix_buffer[0].front(), &m.work_buffer[0].front(), sample_count * sizeof(float));
    memcpy(&mix_buffer[1].front(), &m.work_buffer[1].front(), sample_count * sizeof(float));

    float *plin[] = { &mix_buffer[0].front(), &mix_buffer[1].front() };
    float *plout[] = { &m.work_buffer[0].front(), &m.work_buffer[1].front() };

    if (m.is_muted || m.sequencer_state == sequencer_event_type_mute) {
      m.last_work_audio_result = false;
    } else
      if (m.is_bypassed || m.sequencer_state == sequencer_event_type_thru) {
    m.last_work_audio_result = result;
      } else if ((state == player_state_playing || m.last_work_audio_result)) {
    SETABRPUN(); // turn on flush-to-zero for SSE machines
    m.last_work_audio_result = m.plugin->process_stereo(plin, plout, sample_count, flags);
    // (paniq) flush to zero should be turned off outside our DSP loop
    // because the player library might be running in a process where
    // precise computation is expected (i.e. realtime physics simulation).
    // leaving it on will cause unexpected behavior on code paths
    // we do not control.
    SETGRADUN();       
      }

    std::copy(m.callbacks->feedback_buffer[0].begin() + sample_count, m.callbacks->feedback_buffer[0].begin() + buffer_size, m.callbacks->feedback_buffer[0].begin());
    std::copy(m.callbacks->feedback_buffer[1].begin() + sample_count, m.callbacks->feedback_buffer[1].begin() + buffer_size, m.callbacks->feedback_buffer[1].begin());
    std::copy(m.work_buffer[0].begin(), m.work_buffer[0].begin() + sample_count, m.callbacks->feedback_buffer[0].begin() + buffer_size - sample_count);
    std::copy(m.work_buffer[1].begin(), m.work_buffer[1].begin() + sample_count, m.callbacks->feedback_buffer[1].begin() + buffer_size - sample_count);

    float samplerate = float(master_info.samples_per_second);
    float falloff = std::pow(10.0f, (-48.0f / (samplerate * 20.0f))); // vu meter falloff (-48dB/s)

    if (m.last_work_audio_result) {
      if (scanPeakStereo(&m.work_buffer[0].front(), &m.work_buffer[1].front(), sample_count,
             m.last_work_max_left, m.last_work_max_right, falloff)) {
    // the plugin claims it has generated non-silence, but our scan says otherwise
    m.writemode_errors++;
 
      }

    } else {
      m.last_work_max_left *= std::pow(falloff, sample_count);
      m.last_work_max_right *= std::pow(falloff, sample_count);
    }

The main audio processing code is about in the middle:

m.last_work_audio_result = m.plugin->process_stereo(plin, plout, sample_count, flags);

But you can notice, what might supposed to be considered as denormals check go below this!

I expect you are talking about scanPeakStereo() function. But as I have just said, it goes AFTER the actual plugin call and it does not stops plugin from the work. Thus far it does not reduce any load on a plugin!
As I have noticed in my previous post, m.writemode_errors get's constantly increasing which might also indicate that this "scan" might detect denormals but does not prevent them from being processed by the plugins!
 

[Vytautas Jancauskas]

From what I understand sseoptimization.h takes care of this, by enabling one of these modes: http://software.intel.com/sites/products/documentation/hpc/composerxe/en-us/2011Update/fortran/win/fpops/common/fpops_set_ftz_daz.htm.

It may be outdated though. But the solution of checking the buffer for small values and setting them to zero by hand seems somewhat idiotic to me. Poor machine performance have never been a problem for me so I never looked too much in to it. Well beyond returning proper boolean values from process_stereo.

[Vytautas Jancauskas]

Are denormals even an issue on modern CPUs and compilers?

[Eugene]

Thanks for the article!
It indeed mentions the taking care of denormals.
However, people from #buze channel told me, that denormals have nothing to do with SSE as it's an ordinary x86 (or how they called it?) process.
I will refer this article to them and ask again.

Also yes, most of the plugins do exactly what you said:

checking the buffer for small values and setting them to zero by hand

But I have not yet checked how many exactly are denormals aware.
For instance, MVerb does this!

And YES, it's a HUGE problem for me! For psycle plugins applying/updating denormals filter reduced CPU usage from 70% in idle state to 10%!
And I belive my machine (specs described in the first post) is modern enough!

Also, I have grown up with the rule of having computer in particularly the purposes I need. I now have it for all what I need! And I don't want it to upgrade just because some software doesn't take care of denormals. I'd rather fix the software. Become myself stronger programming wise, keep the environment clean, safe the money for other purposes etc etc.

For me, personally, this is a very principal question!

[Wojciech Aniszewski]

Ok so have you uploaded these changes to bitbucket?

[Eugene]

Not yet.
There are quite a lot of to upload especially psycle-wise.

I wanted to upload altogether when my personal "psycle adaptation project" is successully finished!

But here is the simple fix for MVerb:
http://pastebin.ca/2298112

It tells how to change it's 2 files. This would completely reduce the load during play/stop when machine does not produce any sound! In case of tritium-3.ccm on my machine it's whole 20% down as described above! :)

[Eugene]

It requires quite a lot of efforts to upload to bitbucket, make the pull request.. etc Especially when I already have the fork and it's in development state and I wouldn't like to upload all the files at this stage...

Though I'm intending in completing this very soon as I already have an example on how things are supposed to be solved! :)

[Vytautas Jancauskas]

It is not good enough to upload the files. You need to make a fork of the repository, create a new branch for your changes and make a pull request for that branch. Anything else is completely out of sync with modern collaborative software development standards.

On Mon, Dec 31, 2012 at 2:22 PM, Eugene <webx...@gmail.com> wrote:

It requires quite a lot of efforts to upload to bitbucket, make the pull request.. etc Especially when I already have the fork and it's in development state and I wouldn't like to upload all the files at this stage...

Though I'm intending in completing this very soon as I already have an example on how things are supposed to be solved! :)

[Vytautas Jancauskas]

That is the only way in which I can review the code, suggest changes and for you to implement those changes conveniently.

[Wojciech Aniszewski]

> Anything else is completely out of sync with modern
> collaborative software development standards.

Sermons aside, I made a small change myself - a new .col colour theme, and I think that due to sophistication level of that magical system, I will keep using it locally. I won't make a new branch with my theme:)
regards
w

[Vytautas Jancauskas]

That's fine with me ;) But if you ever feel like sharing it you will have to get accustomed to the sophistication levels of that magical system (which is not all that magical or sophisticated I assure you).

[Vytautas Jancauskas]

This will also hold true if you will want to contribute to any modern open source project, since most of them use git now.

[Vytautas Jancauskas]

A good starting point http://hgbook.red-bean.com/read/

[Wojciech Aniszewski]

Thanks. As you know I develop CFD code on everyday basis, but I admit we're lagging
like 10 years behind everyone in terms of collab. We are now underway with changing
our SVN to GIT:)

What I meant is that I never really used mercurial. I will skip through your link, and
try to upload my theme out there. Mostly because it's simply genius, that's all:)

[Eugene]

Just to keep on track.

I have managed to reduce the load with psycle plugins by twice when compiling them in optimized mode.
But still needs tweaking to keep silent those machines which have to be silent.
Also gotten some suggestions of optmizing whole code by compiling everything into sse2 mode.

Expect to have the next great progress soon so I could finally make a pull request.

[Eugene]

As expected, there is a great progress in resolving these issues!

The example from MVerb fix was used to make the same fix for druttis's Phantom and EQ-3 which reduced the performance for a particular song by 30% !! Which makes the whole program to be very much usable for this song :)

Next machines to fix are Koruz and Arguru Reverb.

For Phantom the fix was a little different, as it's a generator, while for effect type of machines the example fix was taken from Neil's native Lunar Verb.

I'm now feeling myself to fix all of the similar issues with any other necessary machine but thinking to switch to .psy -> .ccm converter.

Will now try to make Neil being also able open .psy files and then also all the various module formats :)

I also expect this to be progressing the same fast and after it's done, "psycle project" would be in a nearly finished state, so that I could open any .psy file which uses native Psycle plugins and play it, I would make a pull request :)

[Vytautas Jancauskas]

The psycle project loading seems to me like extra code to maintain with no real benefits, I will probably not include it in Neil.

[Eugene]

Thanks for letting know.

For me it's one of the main benefits! 

I'll figure something out.

[bluey]

Will these psycle machines be ported(?) to neil somehow?

Vytautas, are you planning to release any updates to neil? :)

[Vytautas Jancauskas]

Yeah, soon. Don't worry. I have a lot of stuff to deal with right now.

[Eugene Babin]

But why porting, as they work already via psy2zzub virtual plugin? I've just made them sound more decent and correct :)

On Mon, Jan 21, 2013 at 3:11 PM, bluey <bsa...@gmail.com> wrote:

[bluey]

I meant porting in the sense of them being included by default, is that possible, or would it make any sense? If they work well enough, and are stable, having more available machines in a standard install would be nice.

[Евгений Бабин]

Ah, then I see the the next sense points in this:

1. Porting them would most likely make more stable and with a with a better performance.

2. They would be always available. Currently, last 2 times (within less than a year frame) I had to contact Psycle developers to fix libboost dependencies issues.

But I can't say much about their value as I am lacking of music production experience.

I have only discovered that there are many .psy examples available that use only native machines and some in real psy/goa trance style which I am trying to make by all this.

Again, personally, just following this path, I am planning to port/expand/enhance/improve some most advanced machines because I have already started exploring music with them but not totally sure were would this lead.. :)