When a sketch runs out of memory, there is no warning, just random malfunctions, nice.

[simmunity]

I really wish there was a way to enable a out of memory error message back to the serial port, or at least cause a reset and known controlled behavior when a sketch runs out of SRAM. Been hunting down a bug for two days and it turns out it is a silent corruption caused by out of memory. Really!!! How can anyone use and trust this stuff with anything important if on memory full, silent corruption and random erroneous execution occurs, the WORST type of problem. This is a JOKE!!! thanks a lot.

[shiftleftplusone]

In C, the programmer is responsible for anything, even for sufficient memory.
If you make C write into the Nirwana, C will do it ! ;)
So I am using a memory check in case I am uncertain about available memory.
In case memory is too small you'll see negative values.
Would this help you?:

//==============================================================================
// sys mem test
//==============================================================================

extern char _end;
extern "C" char *sbrk(int i);
char *ramstart=(char *)0x20070000;
char *ramend=(char *)0x20088000;

//-------------------------------------------------------------------------------------
void memtest() {
char sbuf[64]; // output string

char *heapend=sbrk(0);
register char * stack_ptr asm ("sp");
struct mallinfo mi=mallinfo();
sprintf(sbuf, "Dyn.RAM used: %-10ld ", mi.uordblks);
Serial.println(sbuf);

sprintf(sbuf, "Prg.stat.RAM used %-10ld ", & _end - ramstart);
Serial.println(sbuf);

sprintf(sbuf, "Stack RAM used %-10ld ", ramend - stack_ptr);
Serial.println(sbuf);

sprintf(sbuf, "Free mem: %-10ld ", stack_ptr - heapend + mi.fordblks);
Serial.println(sbuf);

}

[mikaelpatel]

@simmunity
Are you using the heap (malloc/free, new/delete) or was it a stack overflow? If you are using the heap I suggest that you add a wrapper to always check the return value and that the heap is not out of memory. In the wrapper you can add the error handling.

Are you using an AVR or SAM based board? There is a large difference in the amount of available SRAM. For the smaller MPUs such as the ATmega328P on the Uno, Nano, etc, the best advice is to avoid using the heap all together.

Depending on you background and expectations the Arduino boards can seem very small with very limited resources. C++ style has to be "adjusted" to these limitations.

[simmunity]

MiKael,

I am making a Firmata style firmware for IOT that should run on Uno and up. I am not using any mallocs and just strings in sprint and defined globally, some local variables, no C++ at all, and use of sprint, serial, pin I/O, timers, .etc.

The constant strings that are arguments to sprint are being whacked and overwritten. I am using minimal amounts of stack, no recursion, limited local variables. Just loading the program and making minimal use of the code shows that parameters to sprint being whacked. I am about to write all my own format requirements and literally write everything except serial, pin I/O and timer myself.

I don’t know what is blowing away my string constants, but with 30 years of pro software development under my belt, working on machines this small such as PIC and other micros, the Arduino dev environment now many years old leaves a lot to be desired. Seems like a toy environment only. I am using it because I want to open source the firmware I am providing as part of a larger IOT solution and don’t want my customers to have to deal with anything more difficult than the Arduino IDE, but that is proving to be very difficult with no debugger and corruption that cannot be dealt with.

If the Malloc and sprint and other internal Arduino library code runs into memory ful, why don’t they cause a reset and write some bread crumbs onto the serial port. ANYTHING but just corrupt and continue.

Shannon

From: Mikael Patel [mailto:[email protected]]
Sent: Wednesday, July 15, 2015 10:09 AM
To: arduino/Arduino
Cc: simmunity
Subject: Re: [Arduino] When a sketch runs out of memory, there is no warning, just random malfunctions, nice. (#3533)

@simmunity https://github.com/simmunity
Are you using the heap (malloc/free, new/delete) or was it a stack overflow? If you are using the heap I suggest that you add a wrapper to always check the return value and that the heap is not out of memory. In the wrapper you can add the error handling.

Are you using an AVR or SAM based board? There is a large difference in the amount of available SRAM. For the smaller MPUs such as the ATmega328P on the Uno, Nano, etc, the best advice is to avoid using the heap all together.

Depending on you background and expectations the Arduino boards can seem very small with very limited resources. C++ style has to be "adjusted" to these limitations.

—
Reply to this email directly or view it on GitHub #3533 (comment) . https://github.com/notifications/beacon/AFnh5JWULJhJCpPE0KH7yqF6UlDyeXoBks5odotHgaJpZM4FZMq0.gif

[simmunity]

Vogon,

I am making a Firmata style firmware for IOT that should run on Uno and up. I am not using any mallocs and just strings in sprint and defined globally, some local variables, no C++ at all, and use of sprint, serial, pin I/O, timers, .etc.

The constant strings that are arguments to sprint are being whacked and overwritten. I am using minimal amounts of stack, no recursion, limited local variables. Just loading the program and making minimal use of the code shows that parameters to sprint being whacked. I am about to write all my own format requirements and literally write everything except serial, pin I/O and timer myself.

I don’t know what is blowing away my string constants, but with 30 years of pro software development under my belt, working on machines this small such as PIC and other micros, the Arduino dev environment now many years old leaves a lot to be desired. Seems like a toy environment only. I am using it because I want to open source the firmware I am providing as part of a larger IOT solution and don’t want my customers to have to deal with anything more difficult than the Arduino IDE, but that is proving to be very difficult with no debugger and corruption that cannot be dealt with.

If the Malloc and sprint and other internal Arduino library code runs into memory ful, why don’t they cause a reset and write some bread crumbs onto the serial port. ANYTHING but just corrupt and continue.

Shannon

From: VogonJeltz [mailto:[email protected]]
Sent: Wednesday, July 15, 2015 10:06 AM
To: arduino/Arduino
Cc: simmunity
Subject: Re: [Arduino] When a sketch runs out of memory, there is no warning, just random malfunctions, nice. (#3533)

I am using a memory check in case I am uncertain about available memory.
In case memory is too small you'll see negative values.
Would this help you?:

//=====================================================================================
// sys mem test
//=====================================================================================

extern char _end;
extern "C" char *sbrk(int i);
char *ramstart=(char *)0x20070000;
char *ramend=(char *)0x20088000;

//-------------------------------------------------------------------------------------
void memtest() {
char sbuf[128]; // output string

char *heapend=sbrk(0);
register char * stack_ptr asm ("sp");
struct mallinfo mi=mallinfo();
sprintf(sbuf, "Dyn.RAM used: %-10ld ", mi.uordblks);
Serial.println(sbuf);

sprintf(sbuf, "Prg.stat.RAM used %-10ld ", & _end - ramstart);
Serial.println(sbuf);

sprintf(sbuf, "Stack RAM used %-10ld ", ramend - stack_ptr);
Serial.println(sbuf);

sprintf(sbuf, "Free mem: %-10ld ", stack_ptr - heapend + mi.fordblks);
Serial.println(sbuf);

}

—
Reply to this email directly or view it on GitHub #3533 (comment) . https://github.com/notifications/beacon/AFnh5BSVKw-QzlEaseo3m5S9c2nEFTWXks5odoqUgaJpZM4FZMq0.gif

[shiftleftplusone]

ok, I see, I'didn't know that you're already a very advanced programmer and for which target MCUs and projects you are programming. I myself am just programming for the Due or (at least) for the Mega, and I nevertheless have to care about running out of memory.

[mikaelpatel]

@simmunity
You are taking a large risk using the Arduino source code. It is mostly developed by students from Media and Design Curriculum not Computer Science or Software Engineering. The code is very poor quality (measured with basic SQA tools) AND the IDE with preprocessor is for absolute beginners NOT professionals. There is not architecture. There is no design for device driver integration. The base core does not scale and most serious developers have a hard time getting different libraries to work together.

To give you an idea about what you are up against look at the HardwareSerial implementation and compare with my design in Cosa. You can find it here on github. https://github.com/mikaelpatel/Cosa
the UART interface is here https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/IOStream/Driver/UART.hh

In any case without looking at your code it is difficult to say anything more detailed than "always put string literals in program memory" and "use the program memory pointer version of string functions".
Please see

1. http://www.atmel.com/webdoc/AVRLibcReferenceManual/group__avr__stdio_1ga2b829d696b17dedbf181cd5dc4d7a31d.html
2. http://www.atmel.com/webdoc/AVRLibcReferenceManual/group__avr__pgmspace_1ga05ca900ebf7cd121be73c654d9ccb3eb.html

[simmunity]

Mikael,

I had no idea the Arduino environment was this poor. I’ve been working with PIC 18F and 16F stuff and others since 2005 using CCS compiler and never had this type of issue. My co-founder wanted me to create the open source firmware (like fermata) for IOT Inventor using the Arduino tools so others could easily compile it from source. But your now the second person to confirm that this plan is DOA. I am planning to rewrite (mostly just port the majority of the pure ‘C’ code) to the GNU AVR compiler and not depend on any library code I cannot verify as good, mostly to be avoided apparently.

The code I wrote for Arduino mostly deals for setting pin modes and getting and setting values for a variety of devices and dealing with time, interrupts and I2C. The firmware communicates by serial (CRC16 checked framed packets} that are sent over serial to a gateway (a Pi or other larger computer) that talks to the Arduino as a master then bridges the data to the cloud and back to mobile devices all of which is running on a custom stack and programming language and platform. This firmware is just supposed to be simple stuff, getter/setter for a few dozen types of sensors etc. Simple command parsing and response. This is why I was confused and thought I had a bad Mega328 until I caught some bad code stomping on const char format strings in my sprint statements and went, WTF…

You can check out the larger project we are working on at IOTInventor.com and the underlying platform for IOT Inventor at nativecloudsystems.com

I’ll have a look at your serial code and look forward to further chats. Thanks.

Shannon Bailey

co-founder IOT Inventor Inc.

From: Mikael Patel [mailto:[email protected]]
Sent: Wednesday, July 15, 2015 12:36 PM
To: arduino/Arduino
Cc: simmunity
Subject: Re: [Arduino] When a sketch runs out of memory, there is no warning, just random malfunctions, nice. (#3533)

@simmunity https://github.com/simmunity
You are taking a large risk using the Arduino source code. It is mostly developed by students from Media and Design Curriculum not Computer Science or Software Engineering. The code is very poor quality (measured with basic SQA tools) AND the IDE with preprocessor are for absolute beginners NOT professionals. There is not architecture. There is no design for device driver integration. The base core does not scale and most serious developers have a hard time getting different libraries to work together.

To give you an idea about what you are up against look at the HardwareSerial implementation and compare with my design in Cosa. You can find it here on github. https://github.com/mikaelpatel/Cosa
the UART interface is here https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/IOStream/Driver/UART.hh

In any case without looking at your code it is difficult to say anything more detailed than "always put string literals in program memory" and "use the program memory pointer version of string functions".

—
Reply to this email directly or view it on GitHub #3533 (comment) . https://github.com/notifications/beacon/AFnh5OnG3tAKS2v-YWw2xRtN4ppQW70Rks5odq2MgaJpZM4FZMq0.gif

[simmunity]

Vogon,

I’ve created a cross platform cloud/mobile/desktop system at native cloud systems and are using that to make IOT Inventor. So mostly I work on 32 bit+ systems though I am very familiar with PIC 16F and 18F parts and Arm. The reason I am using Arduino is my co-founder wanted me to provide our firmware as open source in an easy to recompile form for beginner customers. But apparently Arduino is a toy environment, not suitable for reliable commercial product development.

Regards,

Shannon Bailey

co-founder IOT Inventor Inc.

From: VogonJeltz [mailto:[email protected]]
Sent: Wednesday, July 15, 2015 12:04 PM
To: arduino/Arduino
Cc: simmunity
Subject: Re: [Arduino] When a sketch runs out of memory, there is no warning, just random malfunctions, nice. (#3533)

ok, I see, I'didn't know that you're already a very advanced programmer and what for which target MCUs and projects you are programming. I myself am just programming for the Due or (at least) for the Mega, and I nevertheless have to care about running out of memory.

—
Reply to this email directly or view it on GitHub #3533 (comment) . https://github.com/notifications/beacon/AFnh5DFSoCyGHFbQQYEqoW0kNeGM-mH1ks5odqYVgaJpZM4FZMq0.gif

[matthijskooijman]

@simmunity The problem with detecting out-of-memory conditions due to stack overflow is that they might occur at any time (mostly on function calls and after function initialization, but also at other times). Check for overflow at all these moments would be terribly inefficient, and I'm not sure if gcc even has a feature to add in the needed coded for this. You can always add your own checks, based on the code posted in one of the comments.

Regarding malloc, AFAIK only the String class in Arduino uses it, and that handles out -of-memory errors somewhat properly (by aborting the current operation). Reporting an error through Serial would not be appropriate - it's not a given that there is actually a serial console on the other end (there might be a serial device attached, or some program that would be confused by a debug message).

As suggested, putting things in flash might free up some memory, leaving more for your stack (mostly strings, using PROGMEM and F()).

@mikaelpatel, Even though I agree that the Arduino code isn't perfect, and the API makes tradeoffs with the novice users in mind, saying the code is "very poor quality" is overdrawn and even slightly offensive.

Suggesting @simmunity to use Cosa or raw avr-gcc does seem like decent advice, given his needs and skill level (though that is pretty much orthogonal to the original subject of this issue).

Since I think the original issue has been answered and both background and possible solutions have been provided, I'm closing this issue. If anyone has more to add, or thinks some changes are required, feel free to leave comments still.

[mikaelpatel]

@matthijskooijman
Please excuse the "very poor quality" comment. It was too harsh even though I would like to see some improvements. There has never been a discussion, study or publication about the API and trade offs for novices!!?? I do believe it is not as easy to understand as often claimed. If you are interested in my professional opinion I am willing to discuss this on the developers forum. Please bring it up there and I will contribute. Cosa is my way of trying to show what industry strength software development is all about even in small scale embedded.

Arduino is a great makers movement! Making the transition to products easier should be part of the requirements. The core software must support low-power, simple multi-tasking, and the source code should be possible to use as teaching examples. This is not the case right now??

[matthijskooijman]

@mikaelpatel, I believe we pretty much agree, I was mostly responding to your wording. Thanks for responding :-)

[lmihalkovic]

 the Arduino dev environment now many years old leaves a lot to be desired. Seems like a toy environment only

I believe this is not fate and lots can be done to if not cure, at least improve some of the symptoms. I think it is safe to say without insulting anyone that the IDE was mostly worked on by people with tremendous good-will and imagination, matched with far less large scale software development experience.

[PaulStoffregen]

...with far less large scale software development experience.

Perhaps you should demonstrate your own software development experience?

You have written a tremendous number of messages, but your github activity log shows almost no code commits. The small amount of code you've shared is of little substance.

If you really do have the skills and experience to match your rhetoric, prove it by sharing code you've actually written. I personally will not take your comments seriously until you do this. I doubt anyone else will either.

[lmihalkovic]

@PaulStoffregen you no doubt realize that regardless of whether or not you can access it, my fork does or does not exist at this very moment in time, and irrespective of how much arrogance, agressiveness or other posturing you use in you comments towards me. I would very much appreciate if you'd keep ignoring me, and in exchange I will extend to you the same neutral indifference.

[PaulStoffregen]

You've recently written dozens of aggressive & strikingly arrogant messages, today even saying the Arduino developers lack large software development experience. Then you call me arrogant for calling attention to your lack of software contributions. You've not demonstrated any experience (and the Arduino devs have published an incredibly successful project used by hundreds of thousands of people). Your hypocrisy is indeed bold.

Nobody is going to take you seriously when you conduct yourself in this poor manner.

[lmihalkovic]

It's unfortunate that you view your current lack of access to my code in such a negative light. I air on the side of considering the degree of truthfullness of any statement irrespective of its author, and do not deny the right to anyone to state an opinion in the name of meeting or not some deserving notion, but that's just me.

I read again your recent long review of the evolution of the Arduino IDE over the last few years (in the devs mailing list), and I realized how far appart we are. I come from working on 200KLOC to 2.5MLOC/40,000 source files projects. From that vantage point this is by all means a small project with a codebase that is more obfuscated than it is complex. And certainly a great platform for building capabilities without sacrificing the simplicity that made its success.

[simmunity]

So I wrote a replacement for sprintf and my own hex, and 16 and 32 signed and unsigned binary to decimal converters but not float (still use dstrtof). I am still getting const char string corruption on an Uno after this upgrade. I then removed the wire library code temporarily to reduce code size and ram useage, reduced the number of servos and related to a minimum and still corruption. There clearly is a bug in code that runs very early in the process, possibly the Serial code or loader code. I sure wish there was a minimal debugger that was part of the arduino IDE as the bug is repeatable. I am posting the firmware for others to play with even though it is pre-alpha release and a bit of a mess in areas due to it being a tiny part of a much larger development and me tacking on extra device support every few weeks. I'd love for the bug to be in my code, but after a lot of rewrite, just don't see where this goes wrong.

So I've posted the pre-alpha version of the firmware and two dependent libraries up on Github at https://github.com/simmunity/IOTI_Firmware.git
so others can have a look and play with the code and maybe see where the corruption is occurring. I haven't stared at the libraries so maybe they are the culprit, or maybe I am just doing something unforgivably stupid.

Thanks and joy,

Shannon