It turns out that if you do a linear search for an unused file name whenever you create a process then launching N processes takes O(N^2) operations. A good rule of thumb is that O(N^2) algorithms are too slow unless you can guarantee that N always stays quite small.
You can detect this specific problem by looking in %userprofile%\appverifierlogs for .dat files. You can detect process creation slowdowns more generally by grabbing an ETW trace, and now you know one more thing to look for.
With log file creation disabled I found that my tracked processes started about three times faster (!) than at the beginning of my test, and the slowdown is completely avoided. This allows 7,000 Application Verifier monitored processes to be spawned in 1.5 minutes, instead of 40 minutes. With my simple test batch file and simple process I see these process-creation rates:
Microsoft could fix this problem by using something other than a monotonically increasing log-file number. If they used the current date and time (to millisecond or higher resolution) as part of the file name then they would get log file names that were more semantically meaningful, and could be created extremely quickly with virtually no unique-file-search logic.
I have upgraded a number crunching application to a multi-threaded program, using the C++11 facilities. It works well on Mac OS X but does not benefit from multithreading on Windows (Visual Studio 2013). Using the following toy program
It is a well-known fact that creating threads is an expensive operation. This is especially true under Windows (used to be true under Linux prior to clone as well).
Also, creating and joining a thread is necessarily slow and does not tell a lot about creating a thread as such. Joining presumes that the thread has exited, which can only happen after it has been scheduled to run. Thus, your measurements include delays introduced by scheduling. Insofar, the times you measure are actually pretty good (they could easily be 20 times longer!).
Creating 20,000 threads like in your benchmark in a real program is a serious error. While it is not strictly illegal or disallowed to create thousands (even millions) of threads, the "correct" way of using threads is to create no more threads than there are approximately CPU cores. One does not create very short-lived threads all the time either.
You might have a few short-lived ones, and you might create a few extra threads (which e.g. block on I/O), but you will not want to create hundreds or thousands of these. Every additional thread (beyond the number of CPU cores) means more context switches, more scheduler work, more cache pressure, and 1MB of address space and 64kB of physical memory gone per thread (due to stack reserve and commit granularity).
Now, assume you create for example 10 threads at program start, it does not matter at all whether this takes 3 milliseconds alltogether. It takes several hundred milliseconds (at least) for the program to start up anyway, nobody will notice a difference.
Visual C++ uses Concurrency Runtime (MS specific) to implement std.thread features. When you directly call any Concurrency Runtime feature/function, it creates a default runtime object (not going into details). Or, when you call std.thread function, it does the same as of ConcRT function was invoked.
The creation of default runtime (or say, scheduler) takes sometime, and hence it appear to be taking sometime. Try creating a std::thread object, let it run; and then execute the benching marking code (whole of above code, for example).
I am considering buying my first Mac, which will be used in a predominantly Windows environment, with Windows fileshares. I need to be able to prevent the creation of .trashes, .ds_store and ._ files on those shares.
I know there are both free and paid utilities that automatically clean these hidden files up on network drives, but I need to actually prevent their creation, because their existing at all, even for just a short time, will interfere with real-time backups running on the filestore.
I don't control all of the shares that I'd want to access, so I do need to do this from the client rather than the servers. So, is there a way that I can use the veto command you describe within MacOS X to prevent the files being written to the server?
The hidden files, if written to the backup, wouldn't interfere with the backup in the sense that they would prevent it from working -- they'd just be written to the backup unecessarily, and since some of the backups run online, I don't want to be uploading more data than is necessary. They also look a mess when browsing from a Windows machine!
they'd just be written to the backup unecessarily, and since some of the backups run online, I don't want to be uploading more data than is necessary. They also look a mess when browsing from a Windows machine!
We need to establish that the version of samba supports the veto options. You will be able to change this. The protocol is called SMB, or windows shares, Samba is the software that lets you access window shares. You need administrative access to your computer. By default the first user on the system gets to be administrator.
Mac os x samba sets the windows hidden bit so the files do not appear to windows users. Most of the files are small. I realize an administrator may get annoyed by seeing the files on a server, but I do not see them causing a technical problem with a backup.
Why don't you link to the drives read only avoids these problems. Just link write when you need to modify a file. You could write an applescript to link the file write then do the copy then change to read only.
Get a virtual machine app such as Parallels. Run a copy of windows in the virtual machine. Connect to the shares on the mac as read only. Connect to the shares in Windows read/write. With the correct configuration, I believe you can get only windows capable of having read/write. With the appropriate configuration of the Windows virtual machine in parallels you can share a folder between the mac and Windows. Thus, to update a file on a windows server, you go into the windows virtual machine and update the file on the server. You can configure Parallels to mix a windows window with a mac windows.
In this post, we will take a deeper dive and first look at how to enable process creation events, then explore a number of examples that describe how process creation events provide valuable information that can be used to achieve the following two broad goals:
In addition to enabling process creation events, another separate setting must be enabled: Include Command Line in Process Creation Events. This setting will include the command line arguments in the logged event.
Oftentimes when a program is started, there are parameters (also known as arguments) passed to the program to tell it what to do. These parameters are of critical importance for defenders because they can give solid clues as to the nature of the activity.
The Include Command Line in Process Creation Events setting is helpfully located in a different place in the Group Policy Management Editor (thanks, Microsoft!). This can be enabled at Administrative Templates > System > Audit Process Creation > set Include Command Line in Process Creation Events to enabled.
The example will be a built-in Windows executable which is used to manipulate the scheduled tasks on a Windows computer. Malware and attackers will often use scheduled tasks to establish persistence, update settings, or to run other tools.
The schtasks.exe program is very commonly executed within a Windows environment for operational purposes. If only the process name is shown in the event, there is no way for a defender to determine if its use is malicious or benign. Including the command line will provide much greater context.
The next example shows the same process, but launched by a sample from a cryptocurrency stealing malware campaign known as Razy. This event now includes the command line arguments used to start the program. This shows the exact command string executed by the malware:
Not only does it tell defenders exactly when the scheduled task will execute, it will tell them what it does, and also provide indicators which can fuel further searches. In this particular example, the malware created this scheduled task to run a script to disable Windows Defender. Malware will occasionally attempt to disable antivirus tools such as Windows Defender to allow other payloads to execute.
We shall now look at some more examples to demonstrate how process creation events with command line can be used to detect and track malware activity. Very often, when malware is launched, it will consist of its own independent process. This is truer for older malware samples.
The malware process itself can be automatically launched in a number of ways, depending on the mechanism of initial infection. For example, shown below is an example of a process created from a sample of HiddenTear ransomware. The process name is a meaningless string of random characters, which possibly changes on each infection:
By enabling process creation events, organizations can log instances of malware processes starting, use these to trace malware execution on the local machine, and identify any possible spreading to other endpoints in the environment.
However, if process creation events with command line are enabled, the event will contain enough information necessary for a defender to not only declare this process as malicious, but to identify the malware as being a type of cryptocurrency miner.
Furthermore, the command line includes a URL as an indicator of compromise which would assist the defender in performing further searches to determine the scope of the infection within the environment. And lastly, the command line parameters in this sample even identify the cryptocurrency coin type being mined (Monero).
The cryptocurrency mining malware in the example above leveraged a built-in Windows program svchost.exe, which in this context, is known as a LOLBIN (living off the land binary) or LOLBAS (living off the land binaries and scripts).
d3342ee215