Often, applications that are free from memory leaks but frequently allocate and delocate dynamic memory show gradual performance degradation in performance if they are kept running for long periods, typical examples are Firefox browser, the Windows OS itself. Finally, they crash. Why is this? Recurrent allocation and deallocation of dynamic memory causes heap fragmentation, especially if the application allocates small memory chunks. A fragmented heap might have many free blocks, but these blocks are small and non-contiguous. To demonstrate this, look at the following scheme that represents the system's heap. Zeros show free memory blocks and ones show memory blocks that are in use : 100101010000101010110
The above heap is highly fragmented. Allocating a memory block that contains five units (i.e., five zeros) will fail, although the systems has 12 free units in total. This is because the free memory isn't contiguous. On the other hand, the following heap has less free memory but it's not fragmented: 1111111111000000
What can you do to avoid heap fragmentation? First, use dynamic memory as little as possible. In most cases, you can use static or automatic storage or use STL containers. Secondly, try to allocate and de-allocate large chunks rather than small ones. For example, instead of allocating a single object, allocate an array of objects at once. As a last resort, use a custom memory pool.
Here are some interesting observations about memory defragmentation by one Mr. Pavlov http://blog.pavlov.net/2007/11/10/memory-fragmentation/