[Ltib] How to get an application backtrace

[Chris Westervelt]

Hey I don't know if anyone out there in LTIB land has successfully done this or not but With all of the approaches I have tried off the internet to get a stack backtrace of my crashed application code on an LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also using 'catchsegv' but the output does not make any sense. My code has deliberate SEGV errors to induce the dump code to work but the output is nonsensical. I find it interesting that when running debug with the exact same code (Debug build) the GDB utility does a fine job of showing the stack back trace just the way I would like to see it almost but I have no idea how it does that. Also a common problem with ARM is the back trace count only accounts for 1 entry which is the return from error interrupt (useless duh). Any suggestions I will repay with kindness when I can answer one of your questions.

Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com<http://www.advantor.com/>
Office: [phone]
Mobile: [phone]
Fax: [phone]
Notice of Confidentiality:
This e-mail communication and the attachments hereto, if any, are intended solely for the information and use of the addressee(s) identified above and may contain information which is legally privileged and/or otherwise confidential. If a recipient of this e-mail communication is not an addressee (or an authorized representative of an addressee), such recipient is hereby advised that any review, disclosure, reproduction, re-transmission or other dissemination or use of this e-mail communication (or any information contained herein) is strictly prohibited. If you are not an addressee and have received this e-mail communication in error, please advise the sender of that circumstance either by reply e-mail or by telephone at [phone], immediately delete this e-mail communication from any computer and destroy all physical copies of same.

[Stuart Hughes]

Hi Chris,

I'm not sure if this may help, but depending on gcc version, it has some
backtrace functions you can use. Take a look at this, maybe it can help?

http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace-when-my-gcc-c-app-crashes

Regards, Stuart

Post by Chris Westervelt
Hey I don't know if anyone out there in LTIB land has successfully
done this or not but With all of the approaches I have tried off the
internet to get a stack backtrace of my crashed application code on an
LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also
using 'catchsegv' but the output does not make any sense. My code has
deliberate SEGV errors to induce the dump code to work but the output
is nonsensical. I find it interesting that when running debug with
the exact same code (Debug build) the GDB utility does a fine job of
showing the stack back trace just the way I would like to see it
almost but I have no idea how it does that. Also a common problem
with ARM is the back trace count only accounts for 1 entry which is
the return from error interrupt (useless duh). Any suggestions I will
repay with kindness when I can answer one of your questions.
*Chris Westervelt*
*Senior Product Development Engineer*
*Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826*
*http://www.advantor.com <http://www.advantor.com/>
Office: [phone]*
*Mobile: [phone]
Fax: [phone]*
*Notice of Confidentiality:*
This e-mail communication and the attachments hereto, if any, are
intended solely for the information and use of the addressee(s)
identified above and may contain information which is legally
privileged and/or otherwise confidential. If a recipient of this
e-mail communication is not an addressee (or an authorized
representative of an addressee), such recipient is hereby advised that
any review, disclosure, reproduction, re-transmission or other
dissemination or use of this e-mail communication (or any information
contained herein) is strictly prohibited. If you are not an addressee
and have received this e-mail communication in error, please advise
the sender of that circumstance either by reply e-mail or by telephone
at [phone], immediately delete this e-mail communication from
any computer and destroy all physical copies of same.**
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

[Chris Westervelt]

Stuart,

This was the first page I came across and nothing in it that I tried got the results. Have you done this successfully yourself? If so I don't have a clue what the hell I'm doing wrong since I poured through this link .

Thanks

Chris

From: Stuart Hughes [mailto:***@zee2.com]
Sent: Monday, September 14, 2015 10:09 AM
To: Chris Westervelt
Cc: ***@nongnu.org
Subject: Re: [Ltib] How to get an application backtrace

Hi Chris,

I'm not sure if this may help, but depending on gcc version, it has some backtrace functions you can use. Take a look at this, maybe it can help?

http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace-when-my-gcc-c-app-crashes

Regards, Stuart


On 14/09/15 14:16, Chris Westervelt wrote:
Hey I don't know if anyone out there in LTIB land has successfully done this or not but With all of the approaches I have tried off the internet to get a stack backtrace of my crashed application code on an LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also using 'catchsegv' but the output does not make any sense. My code has deliberate SEGV errors to induce the dump code to work but the output is nonsensical. I find it interesting that when running debug with the exact same code (Debug build) the GDB utility does a fine job of showing the stack back trace just the way I would like to see it almost but I have no idea how it does that. Also a common problem with ARM is the back trace count only accounts for 1 entry which is the return from error interrupt (useless duh). Any suggestions I will repay with kindness when I can answer one of your questions.

Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com<http://www.advantor.com/>
Office: [phone]
Mobile: [phone]
Fax: [phone]
Notice of Confidentiality:
This e-mail communication and the attachments hereto, if any, are intended solely for the information and use of the addressee(s) identified above and may contain information which is legally privileged and/or otherwise confidential. If a recipient of this e-mail communication is not an addressee (or an authorized representative of an addressee), such recipient is hereby advised that any review, disclosure, reproduction, re-transmission or other dissemination or use of this e-mail communication (or any information contained herein) is strictly prohibited. If you are not an addressee and have received this e-mail communication in error, please advise the sender of that circumstance either by reply e-mail or by telephone at [phone], immediately delete this e-mail communication from any computer and destroy all physical copies of same.





_______________________________________________

LTIB home page: http://ltib.org



Ltib mailing list

***@nongnu.org<mailto:***@nongnu.org>

https://lists.nongnu.org/mailman/listinfo/ltib

[Mike Goins]

On Mon, Sep 14, 2015 at 10:39 AM, Chris Westervelt

Post by Chris Westervelt
Stuart,
This was the first page I came across and nothing in it that I tried got the
results. Have you done this successfully yourself? If so I don’t have a
clue what the hell I’m doing wrong since I poured through this link .

We make extensive use of core dumps.

Somewhere in a startup script set a location to dump to, like /tmp/:

echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern

Then in the controlling startup script for a
process(/etc/init.d/S40myprocess) put

ulimit -c unlimited

start()
...

(or issue ulimit command on console before manually launching)

Want for core to show up in tmp, then we are able to gdb the core
right there on the system.

Application should have debug info compiled into it (-g) not be stripped.

Post by Chris Westervelt
Thanks
Chris
Sent: Monday, September 14, 2015 10:09 AM
To: Chris Westervelt
Subject: Re: [Ltib] How to get an application backtrace
Hi Chris,
I'm not sure if this may help, but depending on gcc version, it has some
backtrace functions you can use. Take a look at this, maybe it can help?
http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace-when-my-gcc-c-app-crashes
Regards, Stuart
Hey I don’t know if anyone out there in LTIB land has successfully done this
or not but With all of the approaches I have tried off the internet to get a
stack backtrace of my crashed application code on an LPC3250 CPU and Kernel
2.6.34, nothing has worked. I have tried also using ‘catchsegv’ but the
output does not make any sense. My code has deliberate SEGV errors to
induce the dump code to work but the output is nonsensical. I find it
interesting that when running debug with the exact same code (Debug build)
the GDB utility does a fine job of showing the stack back trace just the way
I would like to see it almost but I have no idea how it does that. Also a
common problem with ARM is the back trace count only accounts for 1 entry
which is the return from error interrupt (useless duh). Any suggestions I
will repay with kindness when I can answer one of your questions.
Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com
Office: [phone]
Mobile: [phone]
Fax: [phone]
This e-mail communication and the attachments hereto, if any, are intended
solely for the information and use of the addressee(s) identified above and
may contain information which is legally privileged and/or otherwise
confidential. If a recipient of this e-mail communication is not an
addressee (or an authorized representative of an addressee), such recipient
is hereby advised that any review, disclosure, reproduction, re-transmission
or other dissemination or use of this e-mail communication (or any
information contained herein) is strictly prohibited. If you are not an
addressee and have received this e-mail communication in error, please
advise the sender of that circumstance either by reply e-mail or by
telephone at [phone], immediately delete this e-mail communication
from any computer and destroy all physical copies of same.
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
h

[Chris Westervelt]

Thanks Mike!

Question, does the core dump reveal a culprit where the stack gets corrupted by an out of bounds array write? Is there anything here that is related to libSegFault.so linking or preloading?

-----Original Message-----
From: ltib-bounces+chris.westervelt=***@nongnu.org [mailto:ltib-bounces+chris.westervelt=***@nongnu.org] On Behalf Of Mike Goins
Sent: Monday, September 14, 2015 3:31 PM
To: ***@nongnu.org
Subject: Re: [Ltib] How to get an application backtrace

Post by Chris Westervelt
Stuart,
This was the first page I came across and nothing in it that I tried
got the results. Have you done this successfully yourself? If so I
don’t have a clue what the hell I’m doing wrong since I poured through this link .

We make extensive use of core dumps.

Somewhere in a startup script set a location to dump to, like /tmp/:

echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern

Then in the controlling startup script for a
process(/etc/init.d/S40myprocess) put

ulimit -c unlimited

start()
...

(or issue ulimit command on console before manually launching)

Want for core to show up in tmp, then we are able to gdb the core right there on the system.

Application should have debug info compiled into it (-g) not be stripped.

Post by Chris Westervelt
Thanks
Chris
Sent: Monday, September 14, 2015 10:09 AM
To: Chris Westervelt
Subject: Re: [Ltib] How to get an application backtrace
Hi Chris,
I'm not sure if this may help, but depending on gcc version, it has
some backtrace functions you can use. Take a look at this, maybe it can help?
http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace-
when-my-gcc-c-app-crashes
Regards, Stuart
Hey I don’t know if anyone out there in LTIB land has successfully
done this or not but With all of the approaches I have tried off the
internet to get a stack backtrace of my crashed application code on an
LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also
using ‘catchsegv’ but the output does not make any sense. My code has
deliberate SEGV errors to induce the dump code to work but the output
is nonsensical. I find it interesting that when running debug with
the exact same code (Debug build) the GDB utility does a fine job of
showing the stack back trace just the way I would like to see it
almost but I have no idea how it does that. Also a common problem
with ARM is the back trace count only accounts for 1 entry which is
the return from error interrupt (useless duh). Any suggestions I will repay with kindness when I can answer one of your questions.
Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com
Office: [phone]
Mobile: [phone]
Fax: [phone]
This e-mail communication and the attachments hereto, if any, are
intended solely for the information and use of the addressee(s)
identified above and may contain information which is legally
privileged and/or otherwise confidential. If a recipient of this
e-mail communication is not an addressee (or an authorized
representative of an addressee), such recipient is hereby advised that
any review, disclosure, reproduction, re-transmission or other
dissemination or use of this e-mail communication (or any information
contained herein) is strictly prohibited. If you are not an addressee
and have received this e-mail communication in error, please advise
the sender of that circumstance either by reply e-mail or by telephone
at [phone], immediately delete this e-mail communication from any computer and destroy all physical copies of same.
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
https://lists.no

[Mike Goins]

On Mon, Sep 14, 2015 at 3:57 PM, Chris Westervelt

Post by Chris Westervelt
Thanks Mike!
Question, does the core dump reveal a culprit where the stack gets corrupted by an out of bounds array write? Is there anything here that is related to libSegFault.so linking or preloading?

No other action needed.

Some stack corruption (or most of them likely) are hard to find since
if you corrupt the stack, the stack can be trashed making it
impossible to track down.

There are other methods to diagnose if you have to fight that one:

I've had seen some newer compilers tell me that I'm going out of
bounds in some code that has been in place for years.

Take the memory usage off the stack and malloc it to use efence.

cppcheck can find some issues.

Post by Chris Westervelt
-----Original Message-----
Sent: Monday, September 14, 2015 3:31 PM
Subject: Re: [Ltib] How to get an application backtrace

Post by Chris Westervelt
Stuart,
This was the first page I came across and nothing in it that I tried
got the results. Have you done this successfully yourself? If so I
don’t have a clue what the hell I’m doing wrong since I poured through this link .

We make extensive use of core dumps.
echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern
Then in the controlling startup script for a
process(/etc/init.d/S40myprocess) put
ulimit -c unlimited
start()
...
(or issue ulimit command on console before manually launching)
Want for core to show up in tmp, then we are able to gdb the core right there on the system.
Application should have debug info compiled into it (-g) not be stripped.

Post by Chris Westervelt
Thanks
Chris
Sent: Monday, September 14, 2015 10:09 AM
To: Chris Westervelt
Subject: Re: [Ltib] How to get an application backtrace
Hi Chris,
I'm not sure if this may help, but depending on gcc version, it has
some backtrace functions you can use. Take a look at this, maybe it can help?
http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace-
when-my-gcc-c-app-crashes
Regards, Stuart
Hey I don’t know if anyone out there in LTIB land has successfully
done this or not but With all of the approaches I have tried off the
internet to get a stack backtrace of my crashed application code on an
LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also
using ‘catchsegv’ but the output does not make any sense. My code has
deliberate SEGV errors to induce the dump code to work but the output
is nonsensical. I find it interesting that when running debug with
the exact same code (Debug build) the GDB utility does a fine job of
showing the stack back trace just the way I would like to see it
almost but I have no idea how it does that. Also a common problem
with ARM is the back trace count only accounts for 1 entry which is
the return from error interrupt (useless duh). Any suggestions I will repay with kindness when I can answer one of your questions.
Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com
Office: [phone]
Mobile: [phone]
Fax: [phone]
This e-mail communication and the attachments hereto, if any, are
intended solely for the information and use of the addressee(s)
identified above and may contain information which is legally
privileged and/or otherwise confidential. If a recipient of this
e-mail communication is not an addressee (or an authorized
representative of an addressee), such recipient is hereby advised that
any review, disclosure, reproduction, re-transmission or other
dissemination or use of this e-mail communication (or any information
contained herein) is strictly prohibited. If you are not an addressee
and have received this e-mail communication in error, please advise
the sender of that circumstance either by reply e-mail or by telephone
at [phone], immediately delete this e-mail communication from any computer and destroy all physical copies of same.
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
https://lists.nongnu.org/ma

[Chris Westervelt]

Mike,

We are working with legacy C code apps that use stack variables and some static and some stack arrays. I assume cppcheck only works with CPP. Do the compilers you have used handle just plane old C code?

Chris

-----Original Message-----
From: ltib-bounces+chris.westervelt=***@nongnu.org [mailto:ltib-bounces+chris.westervelt=***@nongnu.org] On Behalf Of Mike Goins
Sent: Monday, September 14, 2015 4:08 PM
To: ***@nongnu.org
Subject: Re: [Ltib] How to get an application backtrace

Post by Chris Westervelt
Thanks Mike!
Question, does the core dump reveal a culprit where the stack gets corrupted by an out of bounds array write? Is there anything here that is related to libSegFault.so linking or preloading?

No other action needed.

Some stack corruption (or most of them likely) are hard to find since if you corrupt the stack, the stack can be trashed making it impossible to track down.

There are other methods to diagnose if you have to fight that one:

I've had seen some newer compilers tell me that I'm going out of bounds in some code that has been in place for years.

Take the memory usage off the stack and malloc it to use efence.

cppcheck can find some issues.

Post by Chris Westervelt
-----Original Message-----
Behalf Of Mike Goins
Sent: Monday, September 14, 2015 3:31 PM
Subject: Re: [Ltib] How to get an application backtrace

Post by Chris Westervelt
Stuart,
This was the first page I came across and nothing in it that I tried
got the results. Have you done this successfully yourself? If so I
don’t have a clue what the hell I’m doing wrong since I poured through this link .

We make extensive use of core dumps.
echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern
Then in the controlling startup script for a
process(/etc/init.d/S40myprocess) put
ulimit -c unlimited
start()
...
(or issue ulimit command on console before manually launching)
Want for core to show up in tmp, then we are able to gdb the core right there on the system.
Application should have debug info compiled into it (-g) not be stripped.

Post by Chris Westervelt
Thanks
Chris
Sent: Monday, September 14, 2015 10:09 AM
To: Chris Westervelt
Subject: Re: [Ltib] How to get an application backtrace
Hi Chris,
I'm not sure if this may help, but depending on gcc version, it has
some backtrace functions you can use. Take a look at this, maybe it can help?
http://stackoverflow.com/questions/77005/how-to-generate-a-stacktrace
-
when-my-gcc-c-app-crashes
Regards, Stuart
Hey I don’t know if anyone out there in LTIB land has successfully
done this or not but With all of the approaches I have tried off the
internet to get a stack backtrace of my crashed application code on an
LPC3250 CPU and Kernel 2.6.34, nothing has worked. I have tried also
using ‘catchsegv’ but the output does not make any sense. My code
has deliberate SEGV errors to induce the dump code to work but the
output is nonsensical. I find it interesting that when running debug
with the exact same code (Debug build) the GDB utility does a fine
job of showing the stack back trace just the way I would like to see
it almost but I have no idea how it does that. Also a common problem
with ARM is the back trace count only accounts for 1 entry which is
the return from error interrupt (useless duh). Any suggestions I will repay with kindness when I can answer one of your questions.
Chris Westervelt
Senior Product Development Engineer
Advantor Systems.
12612 Challenger Pkwy
Suite 300
Orlando, FL
32826
http://www.advantor.com
Office: [phone]
Mobile: [phone]
Fax: [phone]
This e-mail communication and the attachments hereto, if any, are
intended solely for the information and use of the addressee(s)
identified above and may contain information which is legally
privileged and/or otherwise confidential. If a recipient of this
e-mail communication is not an addressee (or an authorized
representative of an addressee), such recipient is hereby advised
that any review, disclosure, reproduction, re-transmission or other
dissemination or use of this e-mail communication (or any information
contained herein) is strictly prohibited. If you are not an addressee
and have received this e-mail communication in error, please advise
the sender of that circumstance either by reply e-mail or by
telephone at [phone], immediately delete this e-mail communication from any computer and destroy all physical copies of same.
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org
Ltib mailing list
https://lists.nongnu.org/mailman/listinfo/ltib

_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
https://lists.nongnu.org/mailman/listinfo/ltib
_______________________________________________
LTIB home page: http://ltib.org

Ltib mailing list
***@nongnu.org
https://lists.nongnu.org/mailman/li