[mikaraento]

Hi

I'm embedding the python interpreter into our code, and have noticed that stdlib, CSPyInterpreter and the TLS usage in Python are not safe if memory allocation fails in startup.

For example. ~CSPyInterpreter looks like
__ASSERT_DEBUG(_PyThreadState_Current !=PYTHON_TLS->thread_state,
User::Panic(_L("CSPyInterpreter"), 5));
PyEval_RestoreThread(PYTHON_TLS->thread_state);
Py_Finalize();
SPy_globals_finalize();
if (iCloseStdlib) CloseSTDLIB();
iPyheap->Close();
which will of course crash if PYTHON_TLS returns 0 or iPyheap is zero.

Same kind of code appears all over python_globals.cpp when accessing the TLS.

These are quite easy to fix, e.g., my copy of SPy_tls_finalize looks like
void SPy_tls_finalize(int fini_globals)
{
SPy_Tls* ptls = (SPy_Tls*)Dll::Tls();
if (ptls)
delete ptls->thread_locals;
if (fini_globals && ptls)
delete ptls->globals;
delete ptls;
Dll::SetTls(0);
}
now.

It's a little frustrating that Nokia doesn't apply the same testing criteria to their own code they demand from 3rd parties. This code would never pass the LOWMEM tests.

The worst is still the Symbian stdlib tho. The _REENT macro allocates the reent structure and puts it in TLS as necessary. If that code runs out of memory you get a ESTLIB-INIT 0 Panic. I guess the only way to get that to pass LOWMEM is to pre-allocate a chunk of memory in the beginning InitStdio and free it. For testing code using User::__DbgSetAllocFail() I'm having to dig into the private fields of RHeap to raise an appropriate error beforehand...

Just ranting,
Mika

[rexwal]

- Oops wrong thread .. moved -

[simo.salminen]

Originally Posted by mikaraento

For testing code using User::__DbgSetAllocFail() I'm having to dig into the private fields of RHeap to raise an appropriate error beforehand...

Just curious, why you have to do that?

[mikaraento]

(Re: digging into RHeap)

Originally Posted by simo.salminen

Just curious, why you have to do that?

So I'm running a normal out-of-memory test loop:

while (err==KErrNoMemory) {
User::__DbgSetAllocFail(RHeap::EUser, RHeap::EDeterministic, fail_on);
TRAP(err, r.run());
User::__DbgSetAllocFail(RHeap::EUser, RHeap::ENone, 1);
++fail_on;
}

Now if the fail_on is set so that the _REENT macro cannot allocate the memory it needs I get a ESTLIB-INIT 0 panic and my test run stops. What I n eed to do is to not call the _REENT macro in that case, but to call User::LeaveNoMemory myself instea. To know whether that is the case, I need to go into RHeap and check iFailAllocCount and iFailRate:

if (h->iFailRate > 0 &&
(h->iFailAllocCount % h->iFailRate) < h->iFailRate &&
(h->iFailAllocCount % h->iFailRate) > h->iFailRate - 8) {
User::LeaveNoMemory();
}

I just figured out by testing that _REENT will try to allocate memory 7 times.

The other (better) option would be to reset iFailRate before calling _REENT, and set it to what it was + number of allocations afterward.

Mika

[jplauril]

Originally Posted by mikaraento

Hi
I'm embedding the python interpreter into our code, and have noticed that stdlib, CSPyInterpreter and the TLS usage in Python are not safe if memory allocation fails in startup.

Originally Posted by mikaraento

It's a little frustrating that Nokia doesn't apply the same testing criteria to their own code they demand from 3rd parties. This code would never pass the LOWMEM tests.

You are quite right on both counts. Thanks for pointing out that bug.

You're probably cool with us integrating that fix to the next version, licensed under the Apache 2 license like the rest of that file?