GameMonkey Script

GameMonkey Script Forums
It is currently Wed Nov 14, 2018 11:31 am

All times are UTC




Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: FORK memory bug?
PostPosted: Wed May 09, 2012 4:26 am 
Offline

Joined: Thu Nov 03, 2011 1:00 am
Posts: 6
I am running on Win-7 machine, but compiling GameMonkey in the vanilla 32-bit mode.

I am having some very weird out-of-memory-memory issues with the fork behavior, where it would crash on gmThread::Touch(..) on the assert:

Code:
GM_ASSERT(!"GMTHREAD_MAXBYTESIZE exceeded");


The crash would happen kind of randomly during the run-time of my game and couldn't produce a very clean reproducible case. But I decided to investigate this further, and I think I found the possible culprit. I am writing to confirm the case and to ask what possible solutions there are to this problem..

What is going on is when I am forking, it copies the stack memory of the current thread that is calling the fork. The problem is, my game creates and kill several threads during the duration of the game. When I do create a new thread, if it does not malloc a new thread, it grabs an unused thread and resets it. During the case where it resets an unused thread, and the new forked thread has to realloc to fit the size of the old thread.

Problem is, if you keep forking the "same" thread, it needlessly expands (m_size *= 2;) the memory size to handle the GMTHREAD_SLACKSPACE. As it keeps recycling the same thread, the thread memory-size needed when the gmThread::Touch(...) is called during a fork just keeps doubling exponentially (because it adds some GMTHREAD_SLACKSPACE) until it crashes the system with GMTHREAD_MAXBYTESIZE exceeded.

So the problem is the GMTHREAD_SLACKSPACE that keeps causing a thread to grow unnecessarily, so I removed the need for GMTHREAD_SLACKSPACE during FORKing with:

gmThread.cpp
Code:
case BC_FORK :
{
        ...
        // BEFORE: newthr->Touch( m_size - m_base + 2 );
        newthr->Touch( m_size - m_base + 2 - GMTHREAD_SLACKSPACE);



Tested it in my machine and that fix seemed to work fine....


Last edited by funkbot on Wed May 09, 2012 5:21 am, edited 2 times in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: FORK memory bug?
PostPosted: Thu May 10, 2012 1:19 pm 
Offline

Joined: Mon Dec 15, 2003 1:38 pm
Posts: 707
Thanks for reporting and investigating this funkbot. Your description of the issue and proposed fix does make sense (the way Touch() is called and it's implementation causes perpetual growth). I'll try to have a closer look at it myself in the next few days. I did not write the Fork extension so I must plead some ignorance about it.

I gather that although the problem was initially 'intermittent', you did find a way to reproduce it well enough to track it down and have some confidence in the fix?


Top
 Profile  
Reply with quote  
 Post subject: Re: FORK memory bug?
PostPosted: Thu May 10, 2012 3:17 pm 
Offline

Joined: Thu Nov 03, 2011 1:00 am
Posts: 6
Thanks for the quick response Greg.

Here is a small test script that you can use to quickly achieve the crash within about a second:

Code:
global foo = function()
{
   fork
   {
      sleep ( 0.1f );
      print("killed: " + threadId() );
      foo();
   }
};

foo();


As you can see, the same thread is constantly recycled and the size is doubled because the thread's m_size is constantly not enough because it needs to continuously pad for GMTHREAD_SLACKSPACE during its creation. After about 10 iterations or so of the foo() function, it reaches GMTHREAD_MAXBYTESIZE and crashes on the assert "GMTHREAD_MAXBYTESIZE exceeded".

As I mentioned before, subtracting the Touch(...) ammount by GMTHREAD_SLACKSPACE fixes the problem quite cleanly, and the m_size stays constant during each fork (also, it is sort of what I logically expect from the forking behavior). Doing this fix allows the sample script to run forever, with the m_size being constant during every fork.

Code:
     case BC_FORK :
     {
        ...

        // make sure there is enough room
        newthr->Touch( m_size - m_base + 2 - GMTHREAD_SLACKSPACE)


Top
 Profile  
Reply with quote  
 Post subject: Re: FORK memory bug?
PostPosted: Sat May 12, 2012 1:39 am 
Offline

Joined: Mon Dec 15, 2003 1:38 pm
Posts: 707
Excellent work funcbot! Great repro case and your suggested fix is good.
Keep your changes and I'll update my build to incorporate it.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group