Having found the solution I'll share my strategy,
- If debugger is giving you a hard time use prints in the code to find out what is causing the segmentation fault.
- The most common cause for segmentation is dereferencing invalid pointers.
- If the pointer that is invalid is a local variable then most probably there is something wrong in a called function's local variables (insuficient local buffer size)
- If the pointer going wrong is in the heap then you sould also look into leaks in the heap
- Again if debugger isn't allowing proper inspection due to optimization, gradually comment out code in the area of the segfault until the segfault stops.

- In the case where the error isn't systematically reproducable (the worst case ) try using a simple memory manager that wraps memory allocation to keep tags on what is allocated an who is referencing what.