Computational chemistry
is a field where the execution of tasks can
take a lot of time. Especially if huge molecule databases are used:
e.g., to generate conformations or screen them against target
pharmacophore(s), getting the results can take several hours, days or
even months. Scientific and innovative solutions are needed to face
this problem. Since hardware is getting cheaper and cheaper and
processors exhibit more and more cores, execution time of tasks can be
dramatically reduced.
Thus, LigandScout makes use of multiple cores to split up tasks into
multiple small tasks that can be executed in parallel. But there is
more to it than that. LigandScout comes with a whole framework to
execute these small tasks not only on multiple cores on a single
machine, but in a network on multiple computers using all resources
available to speed up execution.
To use computers connected through the network, the so called “
iworker
”
has to run on each machine in the network. The iworker is a stand-alone
application that can be started either in the
command-line or as a
graphical user interface application
. Each iworker
can execute multiple tasks in parallel to take full advantage of
multi-core and/or multi-processor systems. While the iworker is running,
it listens through a network port for incoming
tasks. This port needs to be specified in the
iworker Settings
and
Preferences/Distributed Calculation Settings
.
Make sure that the system administrator has opened that
port! To keep an overview about the iworker in the network, they are monitored by the
Network Monitor
.
Once the iworker receives a task it executes it and sends the
result back to LigandScout. LigandScout collects all the results of
the small tasks and displays the progress and the overall result.
You can stop an iworker whenever you want. Results will never get lost,
even if the machine running the iworker has crashed unexpectedly. Tasks
which were in execution, just when the iworker stopped or
crashed, will simply be restarted on another iworker. Furthermore,
iworkers can be started on the fly. LigandScout will make use of them as
soon as they are started.
To use this opportunity of distributed calculations, you need to enable
this mode in the
Preferences/Distributed Calculation Settings
.
iworkers that are used by LigandScout
can be added or removed in the
list of iworkers
in the Preferences.
If a listed iworker is not running or cannot be connected to,
LigandScout will regularly retry to connect. In the meantime all
available iworkers will be used. Optionally, LigandScout can find iworkers
automatically, if they run on a LAN (Local Area Network) by using
UDP-Multicast (This works only if broadcasting is enabled by the
system adminisitrator!).
To prevent idle time between finished tasks and new incoming tasks,
which would cost performance, the iworker takes
more tasks than it can execute in parallel and puts them into a so
called “
Work Queue
” (queued tasks).
Whenever a task execution has finished the iworker
starts to execute the next task in the queue with no idle time
in between. Preventing idle time as much
as possible increases the execution performance.
The stand-alone GUI
Press on the iworker application icon to start the iworker GUI.
It consists of an information overview about the iworker, option settings
to be adjusted to your needs and a button that starts/stops it.
This information gives an overview of the status of the
iworker. The
CPU Usage
shows the overall processor load
of the machine running the iworker. The number of tasks that are currently processed
are displayed in the
Tasks in Execution
field. The
next field called
Connections
represents the
number of nodes (LigandScout) that are connected to this particular
iworker. The value of the
Queued Tasks
informs you about
the number of tasks in the
Work Queue
.
The
Total Executed Tasks
implies the
number of tasks that are executed so far. Whereas, the
Total Execution Time
is the time that
the iworker spends for executing the tasks. The
Overall Running Time
field shows the sum of idle time and the execution time (time between starting and
stopping the iworker). Note that restarting the iworker does not reset
the number of tasks, total execution time, and overall running time,
but continues the counting.
Pressing the
Options
button, a dialog appears for
changing the iworker settings.
If the
Discoverable
setting is enabled,
the iworker can be detected in the network. If disabled,
the only way to find this iworker is to define it in the
iworker list
.
This list consists of addresses of the iworkers
and are specified in the LigandScout
Preferences/Distributed Calculation Settings chapter
. To set the
maximum number of tasks executed in parallel, just fill an appropriate value
in the
Parallel Tasks
field.
This number is limited to the available number of cores
(e.g. maximal 2 on a dual-core system, maximal 4 on a quad-core system).
In the
Network Interface
field you can select
the network device to use. The
Incoming Port
represents the network port where the iworker listens for incoming tasks.
This port has to be opened from your system administrator (TCP port).
Note that if the options settings of the iworker are changed, during the iworker runs,
the iworker has to be restarted to make these changes active.
The logging messages of the iworker can be shown in the
console or redirected to a specified file. In the drop down box you can
select the logging level which defines how much information of the log
will be provided. Raising the level leads to a more detailed output.
Choose between
off
,
low
,
medium
,
high
and
highest
.
The
Network Monitor
, inetmon can be used to observe the iworkers in the network.
The iworkers can be detected in two ways. You can specify the network addresses of the iworkers in the
appropriate list
of the Network Monitor Preferences.
Secondly, iworkers are automatically found in the LAN when the
Auto-detectable Option
of the iworker is activated and provided that the broadcasting are enabled.
The stand-alone GUI
Press on the Network Monitor application icon to start the stand-alone GUI.
The Network Monitor consists of an
iworker Information Table
, that provides a detailed overview about the iworkers
and a
Log View
for error handling.
iworker Information Table
The
iworker Information Table
shows information
about the current state of all iworkers found in the network. Each row represents
an iworker and displays following information about that iworker:
Network Addresses of the iworkers (
Addresses
),
number of nodes (LigandScout) that are connected to this particular
iworker (
Connections
), number of CPU's (
CPU
), CPU usage of the machine running the iworker (CPU Usage),
number of tasks that are currently executed (Running Jobs),
number of queued tasks (
Queued Jobs
) and
Enabling
Log
to view errors, warnings and other information about the iworker (
Log
).
Note: Disabling the Log decreases the network-traffic.
The
Log View
displays errors, warnings or other useful information about
the iworker. The Log shows only entries of iworkers selected in the iworker
Information Table. If no iworker is selected, entries are shown for all iworkers.
Any selection in the iworker Information Table can be reseted by clicking
on an empty area in the iworker Information Table.
The detail level of the Log can be changed by the three Log Level
buttons "Error", "Warning", and "Info". The first level "Error" shows
only the errors (bold and red). The second level "Warning" illustrates the
warnings (bold) and errors (bold and red). The last level "Info" shows all
information of the iworker including warnings and errors.
Furthermore, a text filter can be set to show only those entries that
contain the specified text. To apply such a filter, type the text into
the filter area and press the
Filter
button.
To disable the text filter press the
Cross
button that is shown when a text filter is
successfully applied to the Log. You can clear the Log by pressing the
Clear
button. Note: Incoming Log entries are kept in temporary files.
These files are deleted when the Network Monitor is closed and whenever
the
Clear
button is pressed. It is a good idea to clear the Log
if the temporary files require too much hard disc space. Never delete
the temporary files manually! If the space in the Log View is exceeded
the Network Monitor automatically scrolls to the last incoming entry.
Automatic scrolling can be enabled or disabled by toggling the
Lock
button.
Network Monitor Preferences
In the Network Monitor Preferences dialog you can customize
the logging settings and manage the list of iworkers in the network.
You can choose if the logging of the message traffic will be shown in the
console or redirected to a specified file. In the drop down box you can
select the logging level which defines how much information of the log
will be provided. Raising the level leads to a more detailed output.
Choose between
off
(no network log messages),
low
(network errors and warnings),
medium
(connection info messages are shown),
high
(high-level protocol messages)
and
highest
(very verbose for debug).
Add or remove iworkers by adding or removing their
network addresses. A network address has the form
HOSTNAME:PORT or IP_ADDRESS:PORT (e.g.: localhost:4283, 135.27.1.119:1894).
HOSTNAME or IP_ADDRESS is the host name or ip-address of the iworker,
and PORT is the TCP-port the iworker is listening
to. (Remember that the system administrator has to open that port on
the machine the iworker is running on!).
|
