Previewloading31

• 5Suppression file syntax

• Valgrind Leak Check
• Valgrind Abr
• Valgrind Memcheck
• Valgrind Download
• Online Valgrind

Introduction

Valgrind is (mostly) a memory error detector for Linux. It's very good at finding leaks and uninitialised variables; unfortunately it's too good, and usually produces a number of false positives. It comes with suppression files which automatically silence some of these; but not enough, and none specific to wxGTK.

• Valgrind, for debugging memory errors In this unit we explore a useful tool, valgrind, which can help you find bugs in programs that involve illegal memory access and memory leaks. We recommend reading this excellent (and brief!) tutorial from Stanford’s CS107 class: Guide to Valgrind (some of the notes below are adapted from this guide).
• May 11, 2019 Valgrind 原理 valgrind 是一个提供了一些 debug 和优化的工具的工具箱，可以使得你的程序减少内存泄漏或者错误访问. Valgrind 默认使用 memcheck 去检查内存问题.
• Jan 04, 2020 Valgrind is a developer tool for C developers used to find memory issues including C memory leak detection. Valgrind uses instrumentation to collect information about allocated and freed memory to gather complete information about memory blocks.

This howto discusses suppression files and how to create your own. It includes a script to simplify the process.

Why I had to do this

I've used valgrind on apps running on wxGTK-2.4.2 upwards, in several distros. Each time there have been at least dozens of false positives, but never enough to be more than a nuisance. The last time was wxGTK-2.8.10 on debian lenny, where the 'minimal' sample gives 1200 errors. That's annoying, but just about manageable.

I'm now (August 2010) using debian testing, squeeze. The same sample gives 249,000 errors! and a real app many more. Presumably the valgrind/default.supp file doesn't match the squeeze libs too well.

Valgrind is an instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

So I've been forced to teach myself how to suppress these errors. Here are the results; but I don't claim to be an expert in any of this. Please correct any mistakes you find.

Valgrind and suppression files

When valgrind runs its default tool, Memcheck, it automatically tries to read a file called $PREFIX/lib/valgrind/default.supp ($PREFIX will normally be /usr). However you can make it use additional suppression files of your choice by adding --suppressions=<filename> to your command-line invocation. You can repeat this up to 100 times, which should be sufficient for most situations ;)

Rather than having to type this each time, it's more sensible to write it to an rc file. Each time it runs, valgrind looks for options in files called ~/.valgrindrc and ./.valgrindrc. I suggest that you put a general wxGTK suppression in the first, and one that's more specific to your current project in the second. Create the files if they don't already exist. So I now have a ~/.valgrindrc containing:

To check that valgrind is actually using the suppression files, run it with the -v option. The list of suppression files read is near the beginning of the output.

How to make a suppression file

Run valgrind as usual, but with the extra option --gen-suppressions=all. This tells valgrind to print a suppression after every error it finds.

Printing the suppressions inline like this means you have to cut/paste each to the suppression file by hand. Doing that would be fine for just a few errors. but 249k of them might take a while.

A more realistic solution is to save the output to a file. You can do this with a standard unix redirection, or by using the --log-file=<filename> option. With large numbers of errors, you also need to tell valgrind to process them all, with the option --error-limit=no. So, for the 'minimal' sample (which is a good place to start) you might write:

When running the sample it's a good idea to use all its features: move the mouse, run the 'About' dialog, resize the sample etc, as these will often produce extra errors.

You now have a file containing the raw output, with the suppressions mingled with the errors and other stuff. Also, as errors are usually multiple, there'll usually be multiple instances of each suppression. So the next step is to pass minimalraw.log through this gawk script which removes the cruft.

'minimal.supp' is now a valid suppression file, so you can live happily ever after, yes? Well, no. I found that the suppressions were too specific, so running the app again resulted in slightly different errors that weren't suppressed. The answer was to make them less specific by using wildcards. Unfortunately I don't think there's any way that this can be automated. I just went through the suppressions looking for:

• libraries referred to in multiple suppressions, that are unlikely to be involved in 'real' errors. These can easily be filtered out with just one suppression, using the '...' frame-level wildcard.
• near duplicates, that can be caught using a 'normal' wildcard.

During this process, I often found it helpful to refer to the error-message/suppression combination in the 'raw' file, especially as a side-effect of the awk script is to change the order of the suppressions. (It would have been much easier if similar suppressions were displayed adjacently, but I can't think of an easy way to do so. Anyone?)

Having done this, I put the 'wildcard' suppressions into a different file and used it to run 'minimal' and regenerate the suppression file. Then looked for more near duplicates... After several such iterations, I just appended the remaining suppressions, and ended up with a file that suppressed robustly all the 'minimal' false positives.

I've posted the resulting file here. The current one was created using wxGTK-2.8.12 on debian wheezy x86_64. If you don't want to write your own, try it. I don't know how well it'll work for you, but it should at least provide a start.

Update: It's now 2013 and many more people are using wx2.9 so I've also created an additional suppression file for wx2.9.5, though it also works for earlier 2.9 versions. It's designed to be used in addition to the 2.8 one, so either merge the two files or add both to your .valgrindrc.

The next step is to repeat the process (while using the 'minimal' suppression file) with the sample(s) most relevant to your app. This results in a second lot of suppressions which can either be added to the first file or used as a separate file, perhaps pointed to in ./.valgrindrc.

Now when you valgrind your app, the false positives should be reduced to reasonable levels. There will probably still be some though, as your app will make calls that the sample didn't. If you want to get rid of them, make yet another suppression file.

Aggressive use of wildcards in suppressions obviously risks filtering out true positives too. AFAICT I haven't done so, and valgrind still catches my mistakes (which of course are all intentional, just testing.. ;) ).

Suppression file syntax

This is described in [1] and [2]. Each suppression is surrounded with curly-braces, and starts with the line: <insert_a_suppression_name_here>. You can replace this with an informative name if you wish, but it's not necessary.

The second line will start with Memcheck:. Immediately following will be the type of error. See the links for all the possibilities, but I found mostly Cond, Addr8, Value8, and Leak. Unfortunately most false positives seem to produce most or all of these error types, and there's no way of combining the types in a single suppression; so each error needs 3 or 4 otherwise-identical suppressions.

The remaining lines start either with fun: for function calls, or obj: for the names of shared objects. Examples are fun:gdk_set_locale and obj:/usr/lib/libgdk-x11-2.0.so.0.2000.1.

Wildcards

The standard wildcard characters * and ? can be used e.g. obj:/usr/lib/libgdk-x11-* would be more generally applicable than the above line.However I found the frame-level wildcard '...' to be much more useful. This matches zero or more fun: or obj: lines. Using it results in a short, simple suppression that catches multiple similar errors.

As my programming errors are unlikely to involve calling libpango or similar, this should be both efficient and safe.

Step by step summary

1. Check you have gawk and md5sum installed; you almost certainly already have.
2. Open a terminal in the 'minimal' sample directory and run make.
3. Copy the parse_valgrind_suppressions.sh script to this directory (ensure it has executable permission). Alternatively you could use its full filepath in 5.
4. Create a raw suppression file: valgrind --leak-check=full --show-reachable=yes --error-limit=no --gen-suppressions=all --log-file=minimalraw.log ./minimal
5. Process the raw data: cat ./minimalraw.log | ./parse_valgrind_suppressions.sh > minimal.supp
6. Examine the resulting files. Select likely candidates for wildcarding, writing the resulting suppressions to another file e.g. wxGTK-2-8-12.supp.
7. Rerun valgrind using that suppression file: valgrind --leak-check=full --show-reachable=yes --error-limit=no --suppressions=./wxGTK-2.8.12.supp --gen-suppressions=all --log-file=minimalraw.log ./minimal
8. Goto 5.
9. Once there aren't too many errors, or none that you dare wildcard, append the remaining unaltered suppressions to wxGTK-2-8-12.supp.
10. Move wxGTK-2-8-12.supp to somewhere sensible and add its filepath to ~/.valgrindrc.
11. If necessary, repeat using a different sample. Point to the resulting file either in ~/.valgrindrc or in ./.valgrindrc as appropriate.
12. Optionally, repeat with wx2.9.? and either append the results to the 2.8 supp, or (as I do) create a separate 2.9 one and use both.

Links

Written by Nate Hardison, Julie Zelenski and Chris Gregg, with modifications by Nick Troccoli

Click here for a walkthrough video.

Valgrind Memcheck is a tool that detects memory leaks and memory errors. Some of the most difficult C bugs come from mismanagement of memory: allocating the wrong size, using an uninitialized pointer, accessing memory after it was freed, overrunning a buffer, and so on. These types of errors are tricky, as they can provide little debugging information, and tracing the observed problem back to underlying root cause can be challenging. Valgrind is here to help!

Valgrind reports two types of issues: memory errors and memory leaks. When a program dynamically allocates memory and forgets to later free it, it creates a leak. A memory leak generally won't cause a program to misbehave, crash, or give wrong answers, and is not an urgent situation. A memory error, on the other hand, is a red alert. Reading uninitialized memory, writing past the end of a piece of memory, accessing freed memory, and other memory errors can have significant consequences. Memory errors should never be treated casually or ignored. Although this guide describes about how to use Valgrind to find both, keep in mind that errors are by far the primary concern, and memory leaks can generally be resolved later.

Like the debugger, Valgrind runs on your executable, so be sure you have compiled an up-to-date copy of your program. Run it like this, for example, if your program is named memoryLeak:

Valgrind will then start up and run the specified program inside of it to examine it. If you need to pass command-line arguments, you can do that as well:

When it finishes, Valgrind will print a summary of its memory usage. If all goes well, it'll look something like this:

This is what you're shooting for: no errors and no leaks. Another useful metric is the number of allocations and total bytes allocated. If these numbers are the same ballpark as our sample (you can run solution under valgrind to get a baseline), you'll know that your memory efficiency is right on target.

Memory errors can be truly evil. The more overt ones cause spectacular crashes, but even then it can be hard to pinpoint how and why the crash came about. More insidiously, a program with a memory error can still seem to work correctly because you manage to get 'lucky' much of the time. After several 'successful' outcomes, you might wishfully write off what appears to be a spurious catastrophic outcome as a figment of your imagination, but depending on luck to get the right answer is not a good strategy. Running under valgrind can help you track down the cause of visible memory errors as well as find lurking errors you don't even yet know about.

Each time valgrind detects an error, it prints information about what it observed. Each item is fairly terse-- the kind of error, the source line of the offending instruction, and a little info about the memory involved, but often it is enough information to direct your attention to the right place. Here is an example of valgrind running on a buggy program:

The ERROR SUMMARY says there is one error, an invalid write of size 1 (byte, that is). The bad write operation was observed at line 58 in myprogram.c. Let's look at the code:

Looks like an instance of the classic strlen + 1 bug. The code doesn't allocate enough space for the '0' character, so when strcpy went to write it at frag[strlen(buffer)], it accessed memory beyond the end of the malloc'ed piece. Despite the code being clearly wrong, it often may appear to 'work' because malloc commonly rounds up the requested size to the nearest multiple of 4 or 8 and that extra space may cover the shortfall. 'Getting away with it' can lead you to a false sense of security about the code being correct. The next run might get a strange crash that you might write off as a fluke. But vigilantly using valgrind can inform you of the error so you can find and fix it, rather than wait for an observed symptom that may be hard to reproduce.

There are different kinds of memory errors that you may see in the valgrind reports. The most common are:

• Invalid read/write of size X The program was observed to read/write X bytes of memory that was invalid. Common causes include accessing beyond the end of a heap block, accessing memory that has been freed, or accessing into an unallocated region such as from use of a uninitialized pointer.
• Use of uninitialised value or Conditional jump or move depends on uninitialised value(s) The program read the value of a memory location that was not previously written to, i.e. uses random junk. The second more specifically indicates the read occurred in the test expression in an if/for/while. Make sure to initialize all of your variables! Remember that just declaring a variable doesn't put anything in its contents--if you want an int to be 0 or a pointer to be NULL, you must explicitly state so. Note that Valgrind will silently allow a program to propagate an uninitialized value along from variable to variable; the complaint will only come when(if) it eventually uses the value which may be far removed from the root of the error. When tracking down an uninitialized value, run Valgrind with the additional flag --track-origins=yes and it will report the entire history of the value back to the origin which can be very helpful.
• Source and destination overlap in memcpy() The program attempted to copy data from one location to another and the range to be read intersects with the range to be written. Transferring data between overlapping regions using memcpy can garble the result; memmove is the correct function to use in such a situation.
• Invalid free() The program attempted to free a non-heap address or free the same block more than once.

Memory errors in your submission can cause all sorts of varied problems (wrong output, crashes, hangs) and will be subject to significant grading deductions. Be sure to swiftly resolve any memory errors by running Valgrind early and often!

You can ask Valgrind to report on memory leaks in addition to errors. When you allocate heap memory, but don't free it, that is called a leak. For a small, short-lived program that runs and immediately exits, leaks are quite harmless, but for a project of larger size and/or longevity, a repeated small leak can eventually add up. For CS107, we will expect you to deallocate all memory at the end of program execution.

For an example, let's take this memoryLeak.c program:

If we compile and run this, here is the output we get:

This seems to be ok, but let's run it under Valgrind, just to be sure:

It's pretty easy to tell when there's a leak: the alloc/free counts don't match up and you get a LEAK SUMMARY section at the end. (note that it says 2 allocs even though we only call malloc once. Why? Because srand/time allocate memory in their implementations, but they free it as well!). Valgrind also gives a little data about each leak -- how many bytes, how many times it happened, and where in the code the original allocation was made. Multiple leaks attributed to the same cause are coalesced into one entry that summarize the total number of bytes across multiple blocks. Here, the program memoryLeak.c requests memory from the heap and then ends without freeing the memory. This is a memory leak, and valgrind correctly finds the leak: 'definitely lost: 4,000 bytes in 1 blocks'

Valgrind categorizes leaks using these terms:

• definitely lost: heap-allocated memory that was never freed to which the program no longer has a pointer. Valgrind knows that you once had the pointer, but have since lost track of it. This memory is definitely orphaned.
• indirectly lost: heap-allocated memory that was never freed to which the only pointers to it also are lost. For example, if you orphan a linked list, the first node would be definitely lost, the subsequent nodes would be indirectly lost.
• possibly lost: heap-allocated memory that was never freed to which valgrind cannot be sure whether there is a pointer or not.
• still reachable: heap-allocated memory that was never freed to which the program still has a pointer at exit.

These categorizations indicate whether the program has retained a pointer to the memory at exit. If the pointer is available, it will be somewhat easier to add the necessary free call, but it doesn't change that the fact that all are leaks-- that is, memory that was heap-allocated and never freed.

Given that leaks generally do not cause bugs, and incorrect deallocation can, we recommend that you do not worry about freeing memory until your program is finished. Then, you can go back and deallocate your memory as appropriate, ensuring correctness at each step.

If you want more information, you need to include the options --leak-check=full and --show-leak-kinds=all

Because we compiled with the -g flag, valgrind is able to tell us exactly where in our program the leak was created:

In other words, on line 9 in memoryLeak.c, we allocated the memory and it was never freed. This can be super-helpful when debugging your programs!

For more information about Valgrind, also check out the full valgrind manual.

My program runs very slowly under Valgrind. Should I be concerned?

No. Valgrind is running your program in a simulated context and monitoring the runtime activity. Depending on how memory-intensive the program is, this extra checking can slow down a program by 2-5x. This is completely expected.

Valgrind says I leaked memory because of a call to malloc() in main(), but I don't call malloc() in main()! What's going on?

This report can also be a result of calling a library function in main() that itself calls malloc() internally. Common examples include fopen() and strdup(). Make sure to fclose any fopened FILE*s, and free any strduped char*s.

My program runs fine and produces correct output but the Valgrind report shows memory errors. Can I ignore these?

No. An error may not have an observable runtime consequence in some situations but that doesn't mean it doesn't exist. The error is a ticking time bomb that can go off at anytime. Ignoring it and counting on your code continuing to 'get lucky' is a risky practice. Make sure your code always runs Valgrind-clean!

Valgrind Leak Check

My valgrind report includes an ominous-looking entry something like this: 'Warning: set address range perms: large range'. What is this and do I need to worry about it?

Valgrind Abr

Valgrind prints this warning when an unusually large memory region is allocated, on suspicion that the size may be so large due to an error. If the intention of your code was to allocate a large block, then all is well.

My valgrind report suggests to rerun with -v for 'counts of suppressed errors'. What are these? Should I worry about them?

Pay no attention. Some library code does unusual things which can trigger reports from Valgrind even when operating correctly. Those errors/leaks are suppressed as they are known to be spurious. The -v flag causes valgrind to provide verbose commentary about its internal handling of these events. You can safely ignore all suppressed events; no need for you to wade through the verbose chatter.

When I run valgrind with no extra arguments, the ERROR SUMMARY says 0 errors, but the exact same run adding the --leak-check option then reports N errors from N contexts. Do I have errors or don't I?

Valgrind Memcheck

With leak-check enabled, each distinct leak found by valgrind is included in the count of errors. Without leak-check enabled (the default), it doesn't enumerate/count leaks, so only actual memory errors are reported in the summary count.

Valgrind Download

I get a 'Permission denied' message when I attempt to run a particular executable under valgrind even though I can run the program normally. How do I fix?

Online Valgrind

Valgrind refuses if you don't have execute permission according to the file mode of the executable. The file mode is mostly irrelevant on our myth systems (the directory-based AFS permissions take precedence), but Valgrind is paying attention anyway. The command ls -l executable_file shows the file mode bits, the x's indicate execute permission for owner/group/other. Use the command chmod a+x executable_file to enable execute permission for all users.