Using threads with IECS

Tip: How to run the Inference Engine in another thread

The Inference Engine architecture provides rudimentary help for running your inference engines in threads besides your main one. Perhaps you were not aware of it, but the TInferenceEngine component has Lock and Unlock methods. You can use these methods to tell the inference engine to temporarily stop or block (while you change its properties or call other methods) and then continue, like so:

InferenceEngine1.Lock;

try

// your code here

finally

InferenceEngine1.Unlock;

end;

The lock and unlock methods use a critical section (TCriticalSection) to temporarily block another thread. The inference engine's run method calls the lock and unlock every step to ensure you don't stomp on any inference engine structures (such as the Agenda) while it is inferring new facts. Of course, this assumes that you call the Lock and Unlock methods as well.

For this example, we are going to make a simple appiclation that executes two TInferenceEngine components in auxiliary threads, leaving the main application thread free. The source code and the expert systems are available from the web site at www.RiverSoftAVG.com/ThreadDemo.zip The expert systems I use in this example are fullmab.ie and wordgame.ie. These two expert systems are ideal because they take more than a couple of seconds to finish so we can see them executing. Note that if you are compiling the thread demo from scratch and you only have the demo version of the Inference Engine Component Suite, the wordgame.ie will not work because it violates the demo limits. You will receive Out Of Memory exceptions. You can, however, modify the demo to load another expert system instead.

Setting Up The Main Form

First, let's set up the main application. Create a new application and drop two TInferenceEngine threads on the form. Drop two TListBox components on the form (to hold the output from the inference engines, you can use a TMemo component as well). Finally, drop a TButton on the form, this button will be used to start or restart the threads.

To run the threads, create a OnClick event handler for the TButton. In here, we will lock the inference engines, load our expert systems and resume the threads if they are stopped. The code below does just that:

procedure TForm1.Button1Click(Sender: TObject);
begin
     ListBox1.Items.Clear;
     ListBox2.Items.Clear;
     // Lock Inference Engine1 and add the .IE file
     with InferenceEngine1 do
     begin
          // Lock the InferenceEngine so that we can modify it without stomping
          // on the other thread
          Lock;
          try
             // Tell the engine to stop when we wake it back up
             Halt := True;
             // Clear the rules, facts, and everything else
             Clear;
             LoadFromFile( 'fullmab.clp' );
             // reset the engine to prepare for inference
             Reset;
          finally
             Unlock;
          end;
     end;
     // Lock Inference Engine2 and add the .IE file
     with InferenceEngine2 do
     begin
          // Lock the InferenceEngine so that we can modify it without stomping
          // on the other thread
          Lock;
          try
             // Tell the engine to stop when we wake it back up
             Halt := True;
             // Clear the rules, facts, and everything else
             Clear;
             LoadFromFile( 'wordgame.clp' );
             // reset the engine to prepare for inference
             Reset;
          finally
             Unlock;
          end;
     end;
     if Thread1.Suspended then Thread1.Resume;
     if Thread2.Suspended then Thread2.Resume;
end;

To save memory, we are going to use another great feature of the Inference Engine Component Suite and share the user functions between the two TInferenceEngine components. Shift-click the two components and empty out the Packages property set. Now drop on the form the TUserPackages we might need, I dropped the TStandardPackage, TPredicatePackage, TStringPackage, TMathPackage, and TMiscPackage. Do NOT set the Engine properties of the user packages, we will set that in the form OnCreate event.

A word of warning... sharing packages is a great feature but be careful of any user functions that store data in the object. The two inference engines could stomp on each others' data. For example, the TPrintOutFunction stores the last text output (to make it available as a prompt for the read functions). I wanted to keep the example simple so the two expert systems we are using don't require user input. This makes sharing every function safe in this case but you do want to be aware of it.

Creating Our Thread object

So, to run an inference engine in another thread, we need to make a TThread descendant. This thread will be responsible for executing the TInferenceEngine you supply it until it is terminated. Create a new thread object using File->New... Thread Object. I named the thread TIEThread.

For this thread, we want to pass to the constructor the TInferenceEngine component to run AND the TListBox to put the output. First, we create two private fields of the object to hold the inference engine and the list box. Here is our constructor so far:

constructor TIEThread.Create(IE: TInferenceEngine; ListBox: TListBox);
begin
     // Create thread suspended
     inherited Create( True );
     // IE is the inference engine to run
     FIE := IE;
end;

Of course, the Execute method is extremely simple:

procedure TIEThread.Execute;
begin
while (not Terminated) do
IE.Run;
end;

To send output to the list box, we need to create an OnPrintOut event handler in the thread and assign it to the inference engine:

procedure TIEThread.InferenceEnginePrintOut(Sender: TObject; OutID,
Text: String);
begin
ListBox.Items.Add( Text );

end;

Modify the constructor for the assign:

BUT WAIT A SECOND!!! We should not directly access VCL object methods and properties from this thread. In fact, when we create the thread, Borland even includes a caution to wrap all access in Synchronize calls. Ok, we can change the InferenceEngineOnPrintOut to call synchronized another thread method which updates the Listbox. That would work, right? Wrong. Unfortunately, this simple fix will not work in the case. To see why, look at our Button1OnClick event handler. To protect the inference engine, the method locks it, changes it, and then unlocks it. However, if the inference engine is already locked, the Lock method call will block the calling thread (in this case, the main thread). If you remember, I also told you the Run method calls the Lock and Unlock method whe executing a step. If the inference engine has locked itself to execute a step and that step calls the PrintOut method, the TIEThread object will try to synchronize (blocking itself) with the main thread. Deadlock! Uh oh.

So now what do we do? The main problem is the printout call. If we were not trying to synchronize with the main thread, no problem. If we put on our thinking caps, we can come up with a solution (probably more than one). The solution I came up with is to add a message passing thread between the TIEThread and the main thread. The TIEThread object will never actually interact with the main thread. Instead, it will push a message onto the communications thread's queue and immediately return. The communications thread will periodically check its queue, and, when it finds a message, pop it off, synchronize with the main thread, and modify the listbox. The secret is that the message queue will use one critical section and the Listbox modification will use the main thread's synchronization. The communication thread will release the pop the queue and release the critical section (allowing the TIEThread object to grab the critical section and push messages onto the queue) BEFORE synchronizing with the main thread.

In the interests of brevity (this tip is already WAY too long), I won't list the TIECommThread source. Please get the source code from the web site. To summarize, the TIECommThread will allocate a TStringList (as our queue), a critical section (to protect access to the queue), and methods to push and pop. The TIEThread object is changed to create and start a TIECommThread (as well as terminate it), and the OnPrintOut method calls the TIECommThread.Push method.

constructor TIEThread.Create(IE: TInferenceEngine; ListBox: TListBox);
begin
     // Create thread suspended
     inherited Create( True );
     Priority := tpLower;
     // create the communications thread
     FCommThread := TIECommThread.Create( ListBox );
     FCommThread.FreeOnTerminate := True;
     // IE is the inference engine to run
     FIE := IE;
     IE.OnPrintOut := InferenceEnginePrintOut;
end;

destructor TIEThread.Destroy;
begin
CommThread.Terminate;
inherited;
end;

procedure TIEThread.InferenceEnginePrintOut(Sender: TObject; OutID,
Text: String);
begin
// pass the message to the communication thread
CommThread.Push( Text );
end;

Wrapping up

Ok, what is next? Way back at the beginning of the tip, I promised you we would add the TUserPackages to the inference engines on form construction. That is exactly what I am doing here. We also need to allocate the two TInferenceEngine threads. Declare two fields, Thread1 and Thread2, of type TIEThread. We will create them in the constructor:

procedure TForm1.FormCreate(Sender: TObject);
begin
     // Add the packages we need to both inference engines,
     // sharing resources and saving memory. Note, in a real application
     // you would NOT want TPrintOutFunction and TReadXXXFunction to share
     // since they access one variable. We are only going to run non-input
     // expert systems so we will be ok
     StandardPackage1.Reasoners.Add( InferenceEngine1 );
     StandardPackage1.Reasoners.Add( InferenceEngine2 );
     MathPackage1.Reasoners.Add( InferenceEngine1 );
     MathPackage1.Reasoners.Add( InferenceEngine2 );
     PredicatePackage1.Reasoners.Add( InferenceEngine1 );
     PredicatePackage1.Reasoners.Add( InferenceEngine2 );
     StringPackage1.Reasoners.Add( InferenceEngine1 );
     StringPackage1.Reasoners.Add( InferenceEngine2 );
     MiscPackage1.Reasoners.Add( InferenceEngine1 );
     MiscPackage1.Reasoners.Add( InferenceEngine2 );
     // Create a thread for each inference engine
     Thread1 := TIEThread.Create( InferenceEngine1, ListBox1 );
     Thread2 := TIEThread.Create( InferenceEngine2, ListBox2 );
end;

Finally, we need terminate the threads. We will put the terminate code in the OnDestroy event:

procedure TForm1.FormClose(Sender: TObject; var Action: TCloseAction);
begin
     Thread1.Terminate;
     Thread2.Terminate;
     // wait for a couple seconds so that the threads can terminate,
     // with a real application, you would probably want to wait on 2 variables
     // that would be set in the thread's OnTerminate event
     Sleep(2000);
end;

Conclusion

Making the inference engine component suite work with multiple threads is certainly possible. However, before starting the work of doing so, perhaps you should ask yourself whether you need to. Making a multi-threaded program can be a big headache, especially to debug. Instead, consider using the TInferenceEngine.Run( 1 ) command. This method allows you to tell the inference engine to execute one step only before returning. Besides allowing you to avoid threading, you also avoid thread balancing issues - starving some threads at the expense of others. You can call the Run(1) method for multiple inference engines and ensure each gets an equal amount of CPU power.

Please keep in mind that this article is meant as a starting point for developing your own applications using the IECE in separate threads. The example has been simplified to make it as short as it can be. The threads do NOT handle exceptions, which would definitely cause problems if they occured (the two IE files in this example don't though). You also would want to consider using only one communication thread to avoid bogging the system down with too many threads.

I hope this article/tip has been helpful. Any questions or comments, please email me at tggrubbNO@SPAMRiverSoftAVG.com

Send mail to webmasterNO@SPAMRiverSoftAVG.com with questions or comments about this web site. Copyright © 2002-2016 RiverSoftAVG Last modified: September 20, 2010

Send mail to webmasterNO@SPAMRiverSoftAVG.com with questions or comments about this web site.
Copyright © 2002-2016 RiverSoftAVG
Last modified: September 20, 2010