MAGMA  1.2.0
MatrixAlgebraonGPUandMulticoreArchitectures
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Groups
Functions
QUARK: QUeuing And Runtime for Kernels

Functions

void * QUARK_Args_List (Quark *quark)
void * QUARK_Args_Pop (void *args_list, void **last_arg)
Quark * QUARK_Setup (int num_threads)
Quark * QUARK_New (int num_threads)
void QUARK_Barrier (Quark *quark)
void QUARK_Waitall (Quark *quark)
void QUARK_Free (Quark *quark)
void QUARK_Delete (Quark *quark)
Task * QUARK_Task_Init (Quark *quark, void(*function)(Quark *), Quark_Task_Flags *task_flags)
void QUARK_Task_Pack_Arg (Quark *quark, Task *task, int arg_size, void *arg_ptr, int arg_flags)
unsigned long long QUARK_Insert_Task_Packed (Quark *quark, Task *task)
unsigned long long QUARK_Insert_Task (Quark *quark, void(*function)(Quark *), Quark_Task_Flags *task_flags,...)
int QUARK_Cancel_Task (Quark *quark, unsigned long long taskid)
void QUARK_Worker_Loop (Quark *quark, int thread_rank)
Quark_Sequence * QUARK_Sequence_Create (Quark *quark)
int QUARK_Sequence_Cancel (Quark *quark, Quark_Sequence *sequence)
Quark_Sequence * QUARK_Sequence_Destroy (Quark *quark, Quark_Sequence *sequence)
int QUARK_Sequence_Wait (Quark *quark, Quark_Sequence *sequence)
Quark_Sequence * QUARK_Get_Sequence (Quark *quark)
char * QUARK_Get_Task_Label (Quark *quark)
Quark_Task_Flags * QUARK_Task_Flag_Set (Quark_Task_Flags *task_flags, int flag, intptr_t val)

Detailed Description

These functions are available from the QUARK library for the scheduling of kernel routines.

Function Documentation

void* QUARK_Args_List ( Quark *  quark)

Return a pointer to the argument list being processed by the current task and worker.

Parameters:
[in]quarkThe scheduler's main data structure.
Returns:
Pointer to the current argument list (icl_list_t *)

Definition at line 398 of file quark.c.

References quark_task_s::args_list, worker_s::current_task_ptr, QUARK_Thread_Rank(), and quark_s::worker.

{
Task *curr_task = quark->worker[QUARK_Thread_Rank(quark)]->current_task_ptr;
assert( curr_task != NULL );
return (void *)curr_task->args_list;
}

Here is the call graph for this function:

void* QUARK_Args_Pop ( void *  args_list,
void **  last_arg 
)

Return a pointer to the next argument. The variable last_arg should be NULL on the first call, then each subsequent call will used last_arg to get the the next argument. The argument list is not actually popped, it is preservered intact.

Parameters:
[in]args_listPointer to the current arguments
inout]last_arg Pointer to the last argument; should be NULL on the first call
Returns:
Pointer to the next argument

Definition at line 420 of file quark.c.

References icl_list_s::data, icl_list_first(), and icl_list_next().

{
icl_list_t *args = (icl_list_t *)args_list;
icl_list_t *node = (icl_list_t *)*last_arg;
void *arg = NULL;
if ( node == NULL ) {
node = icl_list_first( args );
if (node!=NULL) arg = node->data;
} else {
node = icl_list_next( args, node );
if (node!=NULL) arg = node->data;
}
*last_arg = node;
return arg;
}

Here is the call graph for this function:

void QUARK_Barrier ( Quark *  quark)

Called by the master thread. Wait for all the tasks to be completed, then return. The worker tasks will NOT exit from their work loop.

Parameters:
[in,out]quarkThe scheduler's main data structure.

Definition at line 771 of file quark.c.

References quark_s::all_tasks_queued, quark_s::num_tasks, process_completed_tasks(), TRUE, work_main_loop(), and quark_s::worker.

{
quark->all_tasks_queued = TRUE;
while ( quark->num_tasks > 0 ) {
process_completed_tasks(quark);
work_main_loop( quark->worker[0] );
}
}

Here is the call graph for this function:

Here is the caller graph for this function:

int QUARK_Cancel_Task ( Quark *  quark,
unsigned long long  taskid 
)

Called by any thread. Cancel a task that is in the scheduler. This works by simply making the task a NULL task. The scheduler still processes all the standard dependencies for this task, but when it is time to run the actual function, the scheduler does nothing.

Parameters:
[in,out]quarkThe scheduler's main data structure.
[in]taskidThe taskid returned by a QUARK_Insert_Task
Returns:
1 on success.
-1 if the task cannot be found (may already be done and removed).
-2 if the task is aready running, done, or cancelled.

Definition at line 1182 of file quark.c.

References CANCELLED, DONE, quark_task_s::function, icl_hash_find(), pthread_mutex_lock_wrap(), pthread_mutex_unlock_wrap(), RUNNING, quark_task_s::status, quark_task_s::task_mutex, quark_s::task_set, and quark_s::task_set_mutex.

{
pthread_mutex_lock_wrap( &quark->task_set_mutex );
Task *task = icl_hash_find( quark->task_set, &taskid );
if ( task == NULL ) {
pthread_mutex_unlock_wrap( &quark->task_set_mutex );
return -1;
}
pthread_mutex_lock_wrap( &task->task_mutex );
pthread_mutex_unlock_wrap( &quark->task_set_mutex );
if ( task->status==RUNNING || task->status==DONE || task->status==CANCELLED ) {
pthread_mutex_unlock_wrap( &task->task_mutex );
return -2;
}
task->function = NULL;
pthread_mutex_unlock_wrap( &task->task_mutex );
return 1;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void QUARK_Delete ( Quark *  quark)

Called by the master thread. Wait for all tasks to complete, then join/end the worker threads, and clean up all the data structures.

Parameters:
[in,out]quarkThe scheduler's main data structure.

Definition at line 846 of file quark.c.

References quark_s::coresbind, quark_s::num_threads, pthread_attr_destroy(), pthread_join(), QUARK_Free(), quark_topology_finalize(), QUARK_Waitall(), quark_s::thread_attr, worker_s::thread_id, and quark_s::worker.

{
void *exitcodep = NULL;
int i;
QUARK_Waitall( quark );
/* Wait for workers to quit and join threads */
for (i = 1; i < quark->num_threads; i++)
pthread_join(quark->worker[i]->thread_id, &exitcodep);
pthread_attr_destroy( &quark->thread_attr );
/* Destroy specific structures */
if (quark->coresbind) free(quark->coresbind);
quark_topology_finalize();
/* Destroy hash tables, workers and other data structures */
QUARK_Free( quark );
}

Here is the call graph for this function:

Here is the caller graph for this function:

void QUARK_Free ( Quark *  quark)

Called by the master thread. Free all QUARK data structures, this assumes that all usage of QUARK is completed. This interface does not manage, delete or close down the worker threads.

Parameters:
[in,out]quarkThe scheduler's main data structure.

Definition at line 809 of file quark.c.

References quark_s::address_set, quark_s::address_set_mutex, quark_s::completed_tasks, quark_s::dot_dag_enable, quark_s::dot_dag_mutex, icl_hash_destroy(), quark_s::num_threads, pthread_mutex_destroy(), QUARK_Waitall(), quark_s::task_set, quark_s::tasklevel_width, tasklevel_width_max_level, quark_s::worker, and worker_delete().

{
int i;
QUARK_Waitall(quark);
/* Write the level matching/forcing information */
if ( quark->dot_dag_enable ) {
for (i=1; i<tasklevel_width_max_level && quark->tasklevel_width[i]!=0; i++ ) {
fprintf(dot_dag_file, "%d [label=\"%d:%d\"]\n", i, i, quark->tasklevel_width[i] );
fprintf(dot_dag_file, "%d->%d [style=\"invis\"];\n", i-1, i );
}
fprintf(dot_dag_file, "} \n");
}
/* Destroy hash tables, workers and other data structures */
for (i = 1; i < quark->num_threads; i++)
worker_delete( quark->worker[i] );
worker_delete( quark->worker[0] );
if (quark->worker) free(quark->worker);
if (quark->completed_tasks) free(quark->completed_tasks);
icl_hash_destroy(quark->address_set, NULL, NULL);
icl_hash_destroy(quark->task_set, NULL, NULL);
if ( quark->dot_dag_enable ) {
pthread_mutex_destroy(&quark->dot_dag_mutex);
fclose(dot_dag_file);
}
pthread_mutex_destroy(&quark->address_set_mutex);
free(quark);
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark_Sequence* QUARK_Get_Sequence ( Quark *  quark)

For the current thread, in the current task being executed, return the task's sequence value. This is the value provided when the task was Task_Inserted into a sequence.

Parameters:
[in,out]quarkPointer to the scheduler data structure
Returns:
Pointer to sequence data structure

Definition at line 1980 of file quark.c.

References worker_s::current_task_ptr, QUARK_Thread_Rank(), quark_task_s::sequence, and quark_s::worker.

{
Task *curr_task = quark->worker[QUARK_Thread_Rank(quark)]->current_task_ptr;
assert( curr_task != NULL);
return (Quark_Sequence *)curr_task->sequence;
}

Here is the call graph for this function:

char* QUARK_Get_Task_Label ( Quark *  quark)

For the current thread, in the current task being executed, return the task label. This is the value that was optionally provided when the task was Task_Inserted.

Parameters:
[in,out]quarkPointer to the scheduler data structure
Returns:
Pointer to null-terminated label string
NULL if there is no label

Definition at line 1999 of file quark.c.

References worker_s::current_task_ptr, QUARK_Thread_Rank(), quark_task_s::task_label, and quark_s::worker.

{
Task *curr_task = quark->worker[QUARK_Thread_Rank(quark)]->current_task_ptr;
assert( curr_task != NULL);
return (char *)curr_task->task_label;
}

Here is the call graph for this function:

unsigned long long QUARK_Insert_Task ( Quark *  quark,
void(*)(Quark *)  function,
Quark_Task_Flags *  task_flags,
  ... 
)

Called by the master thread. Add a new task to the scheduler, providing the data pointers, sizes, and dependency information. This function provides the main user interface for the user to write data-dependent algorithms.

Parameters:
[in,out]quarkThe scheduler's main data structure.
[in]functionThe function (task) to be executed by the scheduler
[in]task_flagsFlags to specify task behavior
[in]...Triplets of the form, ending with 0 for arg_size. arg_size, arg_ptr, arg_flags where arg_size: int: Size of the argument in bytes (0 cannot be used here) arg_ptr: pointer: Pointer to data or argument arg_flags: int: Flags indicating argument usage and various decorators INPUT, OUTPUT, INOUT, VALUE, NODEP, SCRATCH LOCALITY, ACCUMULATOR, GATHERV TASK_COLOR, TASK_LABEL (special decorators for VALUE) e.g., arg_flags INPUT | LOCALITY | ACCUMULATOR e.g., arg_flags VALUE | TASK_COLOR
Returns:
A long, long integer which can be used to refer to this task (e.g. for cancellation)

Definition at line 1073 of file quark.c.

References QUARK_Insert_Task_Packed(), QUARK_Task_Init(), and QUARK_Task_Pack_Arg().

{
va_list varg_list;
int arg_size;
unsigned long long taskid;
Task *task = QUARK_Task_Init(quark, function, task_flags);
va_start(varg_list, task_flags);
// For each argument
while( (arg_size = va_arg(varg_list, int)) != 0) {
void *arg_ptr = va_arg(varg_list, void *);
int arg_flags = va_arg(varg_list, int);
QUARK_Task_Pack_Arg( quark, task, arg_size, arg_ptr, arg_flags );
}
va_end(varg_list);
taskid = QUARK_Insert_Task_Packed( quark, task );
return taskid ;
}

Here is the call graph for this function:

Here is the caller graph for this function:

unsigned long long QUARK_Insert_Task_Packed ( Quark *  quark,
Task *  task 
)

Called by the master thread. Add a new task to the scheduler, providing the data pointers, sizes, and dependency information. This function provides the main user interface for the user to write data-dependent algorithms.

Parameters:
[in,out]quarkThe scheduler's main data structure.
[in,out]taskThe packed task structure that already has all the arguments associated with the function

Definition at line 986 of file quark.c.

References quark_s::address_set_mutex, quark_s::all_tasks_queued, FALSE, quark_task_s::function, quark_s::high_water_mark, icl_hash_insert(), LIST_INSERT_HEAD, quark_s::num_tasks, process_completed_tasks(), pthread_mutex_lock_asn(), pthread_mutex_lock_wrap(), pthread_mutex_unlock_asn(), pthread_mutex_unlock_wrap(), quark_task_s::ptr_to_task_in_sequence, quark_check_and_queue_ready_task(), QUARK_ERR, quark_insert_task_dependencies(), quark_task_s::sequence, Quark_sequence_s::sequence_mutex, Quark_sequence_s::status, quark_s::task_set, quark_s::task_set_mutex, quark_task_s::taskid, Quark_sequence_s::tasks_in_sequence, ll_list_node_s::val, work_main_loop(), and quark_s::worker.

{
unsigned long long taskid = task->taskid;
/* Track sequence information if it is provided */
if ( task->sequence ) {
/* if ( task->sequence->status == QUARK_ERR ) { */
/* task_delete( quark, task ); */
/* return QUARK_ERR; */
/* } else { */
/* ll_list_node_t *entry = malloc(sizeof(ll_list_node_t)); */
/* entry->val = task->taskid; */
/* ll_list_head_t *headp = task->sequence->tasks_in_sequence; */
/* pthread_mutex_lock_wrap( &task->sequence->sequence_mutex ); */
/* LIST_INSERT_HEAD( headp, entry, entries ); */
/* pthread_mutex_unlock_wrap( &task->sequence->sequence_mutex ); */
/* /\* Keep pointer to task in sequence so it can be deleted when task completes *\/ */
/* task->ptr_to_task_in_sequence = entry; */
/* printf("sequence %p task %ld addto \n", task->sequence, task->taskid ); */
/* } */
/* TODO FIXME */
if ( task->sequence->status == QUARK_ERR )
task->function = NULL;
ll_list_node_t *entry = malloc(sizeof(ll_list_node_t));
entry->val = task->taskid;
ll_list_head_t *headp = task->sequence->tasks_in_sequence;
pthread_mutex_lock_wrap( &task->sequence->sequence_mutex );
LIST_INSERT_HEAD( headp, entry, entries );
pthread_mutex_unlock_wrap( &task->sequence->sequence_mutex );
/* Keep pointer to task in sequence so it can be deleted when task completes */
task->ptr_to_task_in_sequence = entry;
//printf("sequence %p task %ld addto \n", task->sequence, task->taskid );
}
/* Insert the task in the address hash, locking access to the address set hash */
pthread_mutex_lock_asn( &quark->address_set_mutex );
/* For repeated usage of the scheduler, if tasks are being added repeatedly
* then quark->finalize and quark->all_tasks_queued must be set false */
quark->all_tasks_queued = FALSE;
quark_insert_task_dependencies( quark, task );
/* FIXME does this need to be protected */
quark->num_tasks++;
/* Save the task, indexed by its taskid */
pthread_mutex_lock_wrap( &quark->task_set_mutex );
icl_hash_insert( quark->task_set, &task->taskid, task );
pthread_mutex_unlock_wrap( &quark->task_set_mutex );
// Check if the task is ready
quark_check_and_queue_ready_task( quark, task );
pthread_mutex_unlock_asn(&quark->address_set_mutex);
/* If conditions are right, master works; this will return when
* num_tasks becomes less than low_water_mark */
process_completed_tasks(quark);
while (quark->num_tasks >= quark->high_water_mark) {
work_main_loop(quark->worker[0]);
process_completed_tasks(quark);
}
return taskid ;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark* QUARK_New ( int  num_threads)

Called by the master thread. Allocate and initialize the scheduler data stuctures and spawn worker threads. Used when this scheduler is to do all the thread management.

Parameters:
[in]num_threadsNumber of threads to be used (1 master and rest compute workers). If num_threads < 1, first try environment variable QUARK_NUM_THREADS or use use num_threads = number of cores
Returns:
Pointer to the QUARK data structure.

Definition at line 728 of file quark.c.

References quark_s::coresbind, pthread_attr_init(), pthread_attr_setscope(), pthread_create(), PTHREAD_SCOPE_SYSTEM, quark_get_affthreads(), quark_get_numthreads(), quark_setaffinity(), QUARK_Setup(), quark_topology_init(), quark_s::thread_attr, worker_s::thread_id, work_set_affinity_and_call_main_loop(), and quark_s::worker.

{
int i, nthrd;
/* Init number of cores and topology */
quark_topology_init();
/* Get number of threads */
if ( num_threads < 1 ) {
nthrd = quark_get_numthreads();
if ( nthrd == -1 ) {
nthrd = 1;
}
}
else
nthrd = num_threads;
/* Create scheduler data structures for master and workers */
Quark *quark = QUARK_Setup(nthrd);
/* Get binding informations */
quark->coresbind = quark_get_affthreads();
/* Setup thread attributes */
pthread_attr_init(&quark->thread_attr);
/* pthread_setconcurrency(quark->num_threads); */
pthread_attr_setscope(&quark->thread_attr, PTHREAD_SCOPE_SYSTEM);
/* Then start the threads, so that workers can scan the structures easily */
for(i = 1; i < nthrd; i++) {
int rc = pthread_create(&quark->worker[i]->thread_id, &quark->thread_attr, (void *(*)(void *))work_set_affinity_and_call_main_loop, quark->worker[i]);
assert(rc == 0);
}
quark_setaffinity( quark->coresbind[0] );
return quark;
}

Here is the call graph for this function:

Here is the caller graph for this function:

int QUARK_Sequence_Cancel ( Quark *  quark,
Quark_Sequence *  sequence 
)

Can be called by any thread. Cancels all the remaining tasks in a sequence using QUARK_Cancel_Task and changes the state so that future tasks belonging to that sequence are ignored.

Parameters:
[in,out]quarkPointer to the scheduler data structure
[in,out]sequencePointer to a sequence data structure
Returns:
0 (QUARK_SUCCESS) on success
-1 (QUARK_ERR) on failure

Definition at line 1880 of file quark.c.

References LIST_FOREACH_SAFE, pthread_mutex_lock_wrap(), pthread_mutex_unlock_wrap(), QUARK_Cancel_Task(), QUARK_ERR, QUARK_SUCCESS, Quark_sequence_s::sequence_mutex, Quark_sequence_s::status, Quark_sequence_s::tasks_in_sequence, and ll_list_node_s::val.

{
int retval;
if ( quark==NULL || sequence==NULL ) return QUARK_ERR;
pthread_mutex_lock_wrap( &sequence->sequence_mutex );
if ( sequence->status != QUARK_SUCCESS ) {
/* sequence already cancelled */
retval = QUARK_SUCCESS;
} else {
sequence->status = QUARK_ERR;
ll_list_node_t *np, *np_temp;
LIST_FOREACH_SAFE( np, sequence->tasks_in_sequence, entries, np_temp ) {
long long int taskid = np->val;
/* Find taskid, make function NULL */
QUARK_Cancel_Task( quark, taskid );
/* Task node is removed from sequence when it finishes and is
* deleted; or when sequence is destroyed */
}
retval = QUARK_SUCCESS;
}
pthread_mutex_unlock_wrap( &sequence->sequence_mutex );
return retval;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark_Sequence* QUARK_Sequence_Create ( Quark *  quark)

Called by the control program. Creates a new sequence data structure and returns it. This can be used to put a sequence of tasks into a group and cancel that group if an error condition occurs.

Parameters:
[in]outquark Pointer to the scheduler data structure
Returns:
Pointer to the newly created sequence structure.

Definition at line 1853 of file quark.c.

References LIST_INIT, pthread_mutex_init(), QUARK_SUCCESS, Quark_sequence_s::sequence_mutex, Quark_sequence_s::status, and Quark_sequence_s::tasks_in_sequence.

{
Quark_Sequence *sequence = malloc(sizeof(Quark_Sequence));
assert( sequence != NULL );
sequence->status = QUARK_SUCCESS;
pthread_mutex_init( &sequence->sequence_mutex, NULL );
ll_list_head_t *head = malloc(sizeof(ll_list_head_t));
assert ( head != NULL );
LIST_INIT(head);
sequence->tasks_in_sequence = head;
return sequence;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark_Sequence* QUARK_Sequence_Destroy ( Quark *  quark,
Quark_Sequence *  sequence 
)

Called by the control program. Cancels all the remaining tasks in a sequence using QUARK_Cancel_Task and deletes the sequence data structure.

Parameters:
[in,out]quarkPointer to the scheduler data structure
[in,out]sequencePointer to a sequence data structure
Returns:
A NULL pointer; which can be used to reset the sequence structure

Definition at line 1917 of file quark.c.

References LIST_FOREACH_SAFE, LIST_REMOVE, pthread_mutex_destroy(), pthread_mutex_lock_wrap(), pthread_mutex_unlock_wrap(), QUARK_Cancel_Task(), QUARK_Sequence_Wait(), Quark_sequence_s::sequence_mutex, Quark_sequence_s::tasks_in_sequence, and ll_list_node_s::val.

{
if ( quark==NULL || sequence==NULL) return NULL;
//printf("QUARK_Sequence_Destroy %p status %d\n", sequence, sequence->status);
pthread_mutex_lock_wrap( &sequence->sequence_mutex );
ll_list_node_t *np, *np_temp;
ll_list_head_t *head = sequence->tasks_in_sequence;
LIST_FOREACH_SAFE( np, head, entries, np_temp ) {
long long int taskid = np->val;
QUARK_Cancel_Task( quark, taskid );
}
pthread_mutex_unlock_wrap( &sequence->sequence_mutex );
QUARK_Sequence_Wait( quark, sequence );
pthread_mutex_lock_wrap( &sequence->sequence_mutex );
LIST_FOREACH_SAFE( np, head, entries, np_temp ) {
LIST_REMOVE( np, entries );
free( np );
}
pthread_mutex_unlock_wrap( &sequence->sequence_mutex );
free( head );
head = NULL;
pthread_mutex_destroy( &sequence->sequence_mutex );
free( sequence );
sequence = NULL;
return sequence;
}

Here is the call graph for this function:

Here is the caller graph for this function:

int QUARK_Sequence_Wait ( Quark *  quark,
Quark_Sequence *  sequence 
)

Called by the control program. Returns when all the tasks in a sequence have completed.

Parameters:
[in,out]quarkPointer to the scheduler data structure
[in,out]sequencePointer to a sequence structure
Returns:
0 on success
-1 on failure

Definition at line 1957 of file quark.c.

References LIST_EMPTY, process_completed_tasks(), QUARK_ERR, QUARK_SUCCESS, QUARK_Thread_Rank(), Quark_sequence_s::tasks_in_sequence, work_main_loop(), and quark_s::worker.

{
if ( quark==NULL || sequence==NULL) return QUARK_ERR;
int myrank = QUARK_Thread_Rank( quark );
while ( !LIST_EMPTY( sequence->tasks_in_sequence ) ) {
process_completed_tasks( quark );
work_main_loop( quark->worker[myrank] );
}
return QUARK_SUCCESS;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark* QUARK_Setup ( int  num_threads)

Called by the master thread. This routine does not do thread management, so it can be used with a larger libarary. Allocate and initialize the scheduler data stuctures for the master and num_threads worker threads.

Parameters:
[in]num_threadsNumber of threads to be used (1 master and rest compute workers).
Returns:
Pointer to the QUARK scheduler data structure.

Definition at line 653 of file quark.c.

References address_hash_function(), address_key_compare(), quark_s::address_set, quark_s::address_set_mutex, quark_s::all_tasks_queued, quark_s::completed_tasks, quark_s::completed_tasks_mutex, quark_s::dot_dag_enable, DOT_DAG_FILENAME, quark_s::dot_dag_mutex, FALSE, fopen, quark_s::high_water_mark, icl_hash_create(), quark_s::list_robin, quark_s::low_water_mark, quark_s::num_queued_tasks, quark_s::num_queued_tasks_cond, quark_s::num_tasks, quark_s::num_threads, pthread_cond_init(), pthread_mutex_init(), pthread_self(), quark_getenv_int(), QUARK_Thread_Rank(), quark_s::queue_before_computing, quark_s::rank, quark_s::start, TAILQ_INIT, quark_s::task_set, quark_s::task_set_mutex, quark_s::tasklevel_width, tasklevel_width_max_level, worker_s::thread_id, TRUE, ullong_hash_function(), ullong_key_compare(), quark_s::war_dependencies_enable, quark_s::worker, and worker_new().

{
int i = 0;
Quark *quark = (Quark *) malloc(sizeof(Quark));
assert(quark != NULL);
/* Used to tell master when to act as worker */
int quark_unroll_tasks_per_thread = quark_getenv_int("QUARK_UNROLL_TASKS_PER_THREAD", 20);
int quark_unroll_tasks = quark_getenv_int("QUARK_UNROLL_TASKS", quark_unroll_tasks_per_thread * num_threads);
quark->war_dependencies_enable = quark_getenv_int("QUARK_WAR_DEPENDENCIES_ENABLE", 0);
quark->queue_before_computing = quark_getenv_int("QUARK_QUEUE_BEFORE_COMPUTING", 0);
quark->dot_dag_enable = quark_getenv_int("QUARK_DOT_DAG_ENABLE", 0);
if ( quark->dot_dag_enable ) quark->queue_before_computing = 1;
if ( quark->queue_before_computing==1 || quark_unroll_tasks==0 ) {
quark->high_water_mark = (int)(INT_MAX - 1);
quark->low_water_mark = (int)(quark->high_water_mark);
} else {
quark->low_water_mark = (int)(quark_unroll_tasks);
quark->high_water_mark = (int)(quark->low_water_mark + quark->low_water_mark*0.25);
}
quark->num_queued_tasks = 0;
pthread_cond_init( &quark->num_queued_tasks_cond, NULL );
quark->num_threads = num_threads;
quark->list_robin = 0;
quark->start = FALSE;
quark->all_tasks_queued = FALSE;
quark->num_tasks = 0;
quark->task_set = icl_hash_create( 0x1<<12, ullong_hash_function, ullong_key_compare );
pthread_mutex_init( &quark->task_set_mutex, NULL );
/* Define some function pointers that match a C++ interface */
quark->rank = QUARK_Thread_Rank;
/* Create hash table to hold addresses */
quark->address_set = icl_hash_create( 0x01<<12, address_hash_function, address_key_compare);
pthread_mutex_init(&quark->address_set_mutex, NULL);
/* To handle completed tasks */
quark->completed_tasks = malloc(sizeof(completed_tasks_head_t));
assert ( quark->completed_tasks != NULL );
TAILQ_INIT( quark->completed_tasks );
pthread_mutex_init(&quark->completed_tasks_mutex, NULL);
/* Setup workers */
quark->worker = (Worker **) malloc(num_threads * sizeof(Worker *));
assert(quark->worker != NULL);
/* The structure for the 0th worker will be used by the master */
quark->worker[0] = worker_new(quark, 0);
quark->worker[0]->thread_id = pthread_self();
if ( quark->dot_dag_enable ) {
fopen(&dot_dag_file, DOT_DAG_FILENAME, "w"); /* global FILE variable */
fprintf(dot_dag_file, "digraph G { size=\"10,7.5\"; center=1; orientation=portrait; \n");
for (i=0; i<tasklevel_width_max_level; i++ )
quark->tasklevel_width[i] = 0;
pthread_mutex_init(&quark->dot_dag_mutex, NULL);
/* fprintf(dot_dag_file, "%d [label=\"%d %d\",style=\"invis\"]\n", 0, 0, quark->tasklevel_width[i] ); */
fprintf(dot_dag_file, "%d [style=\"invis\"]\n", 0);
}
/* Launch workers; first create the structures */
for(i = 1; i < num_threads; i++)
quark->worker[i] = worker_new(quark, i);
/* Threads can start as soon as they want */
quark->start = TRUE;
return quark;
}

Here is the call graph for this function:

Here is the caller graph for this function:

Quark_Task_Flags* QUARK_Task_Flag_Set ( Quark_Task_Flags *  task_flags,
int  flag,
intptr_t  val 
)

Set various task level flags. This flag data structure is then provided when the task is created/inserted. Each flag can take a value which is either an integer or a pointer. 

     Select from one of the flags:
     TASK_PRIORITY : an integer (0-MAX_INT)
     TASK_LOCK_TO_THREAD : an integer for the thread number
     TASK_LABEL : a string pointer (NULL terminated) for the label
     TASK_COLOR :  a string pointer (NULL terminated) for the color.
     TASK_SEQUENCE : takes pointer to a Quark_Sequence structure
Parameters:
[in,out]flagsPointer to a Quark_Task_Flags structure
[in]flagOne of the flags ( TASK_PRIORITY, TASK_LOCK_TO_THREAD, TASK_LABEL, TASK_COLOR, TASK_SEQUENCE )
[in]valA integer or a pointer value for the flag ( uses the intptr_t )
Returns:
Pointer to the updated Quark_Task_Flags structure

Definition at line 2145 of file quark.c.

References TASK_COLOR, quark_task_flags_s::task_color, TASK_LABEL, quark_task_flags_s::task_label, TASK_LOCK_TO_THREAD, quark_task_flags_s::task_lock_to_thread, TASK_PRIORITY, quark_task_flags_s::task_priority, TASK_SEQUENCE, quark_task_flags_s::task_sequence, TASK_THREAD_COUNT, and quark_task_flags_s::task_thread_count.

{
switch (flag) {
case TASK_PRIORITY:
task_flags->task_priority = (int)val;
break;
case TASK_LOCK_TO_THREAD:
task_flags->task_lock_to_thread = (int)val;
break;
case TASK_LABEL:
task_flags->task_label = (char *)val;
break;
case TASK_COLOR:
task_flags->task_color = (char *)val;
break;
case TASK_SEQUENCE:
task_flags->task_sequence = (Quark_Sequence *)val;
break;
case TASK_THREAD_COUNT:
task_flags->task_thread_count = (int)val;
break;
}
return task_flags;
}

Here is the caller graph for this function:

Task* QUARK_Task_Init ( Quark *  quark,
void(*)(Quark *)  function,
Quark_Task_Flags *  task_flags 
)

Called by the master thread. This is used in argument packing, to create an initial task data structure. Arguments can be packed into this structure, and it can be submitted later.

Parameters:
[in,out]quarkThe scheduler's main data structure.
[in]functionThe function (task) to be executed by the scheduler
[in]task_flagsFlags to specify task behavior

Definition at line 895 of file quark.c.

References quark_task_s::function, quark_set_task_flags_in_task_structure(), and quark_task_new().

{
Task *task = quark_task_new();
task->function = function;
quark_set_task_flags_in_task_structure( quark, task, task_flags );
return task;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void QUARK_Task_Pack_Arg ( Quark *  quark,
Task *  task,
int  arg_size,
void *  arg_ptr,
int  arg_flags 
)

Called by the master thread. This is used in argument packing, to pack/add arguments to a task data structure.

Parameters:
[in,out]quarkThe scheduler's main data structure.
[in,out]taskThe task data struture to hold the arguments
[in]arg_sizeSize of the argument in bytes (0 cannot be used here)
[in]arg_ptrPointer to data or argument
[in]arg_flagsFlags indicating argument usage and various decorators INPUT, OUTPUT, INOUT, VALUE, NODEP, SCRATCH LOCALITY, ACCUMULATOR, GATHERV TASK_COLOR, TASK_LABEL (special decorators for VALUE) e.g., arg_flags INPUT | LOCALITY | ACCUMULATOR e.g., arg_flags VALUE | TASK_COLOR

Definition at line 925 of file quark.c.

References ACCUMULATOR, arg_dup(), quark_task_s::args_list, quark_task_s::dependency_list, dependency_new(), DIRECTION_MASK, quark_s::dot_dag_enable, GATHERV, icl_list_append(), INOUT, INPUT, LOCALITY, quark_task_s::lock_to_thread, quark_task_s::num_dependencies, quark_task_s::num_dependencies_remaining, OUTPUT, quark_task_s::priority, SCRATCH, quark_task_s::scratch_list, scratch_new(), quark_task_s::sequence, TASK_COLOR, quark_task_s::task_color, dependency_s::task_dependency_list_node_ptr, TASK_LABEL, quark_task_s::task_label, TASK_LOCK_TO_THREAD, TASK_PRIORITY, TASK_SEQUENCE, TASK_THREAD_COUNT, quark_task_s::task_thread_count, and VALUE.

{
icl_list_t *task_args_list_node_ptr=NULL;
// extract information from the flags
bool arg_locality = (bool) ((arg_flags & LOCALITY) != 0 );
bool accumulator = (bool) ((arg_flags & ACCUMULATOR) != 0 );
bool gatherv = (bool) ((arg_flags & GATHERV) != 0 );
bool task_priority = (bool) ((arg_flags & TASK_PRIORITY) != 0 );
bool task_lock_to_thread = (bool) ((arg_flags & TASK_LOCK_TO_THREAD) != 0 );
bool task_thread_count = (bool) ((arg_flags & TASK_THREAD_COUNT) != 0 );
bool task_color = (bool) ((arg_flags & TASK_COLOR) != 0 );
bool task_label = (bool) ((arg_flags & TASK_LABEL) != 0 );
bool task_sequence = (bool) ((arg_flags & TASK_SEQUENCE) != 0 );
quark_direction_t arg_direction = (quark_direction_t) (arg_flags & DIRECTION_MASK);
if (arg_direction == VALUE) {
/* If argument is a value; Copy the contents to the argument buffer */
if ( task_priority ) task->priority = *((int *)arg_ptr);
else if ( task_lock_to_thread ) task->lock_to_thread = *((int *)arg_ptr);
else if ( task_thread_count ) task->task_thread_count = *((int *)arg_ptr);
else if ( task_sequence ) task->sequence = *((Quark_Sequence **)arg_ptr);
else if ( task_color && quark->dot_dag_enable ) {
if ( task->task_color && task->task_color!=quark_task_default_color) free(task->task_color);
task->task_color = arg_dup(arg_ptr, arg_size);
}
else if ( task_label && quark->dot_dag_enable ) {
if ( task->task_label && task->task_label!=quark_task_default_label) free(task->task_label);
task->task_label = arg_dup(arg_ptr, arg_size) ;
}
else task_args_list_node_ptr = icl_list_append(task->args_list, arg_dup(arg_ptr, arg_size));
} else {
/* Else - argument is a pointer; Copy the pointer to the argument buffer - pass by reference */
task_args_list_node_ptr = icl_list_append(task->args_list, arg_dup((char *) &arg_ptr, sizeof(char *)));
}
if ((arg_ptr != NULL) && ( arg_direction==INPUT || arg_direction==INOUT || arg_direction==OUTPUT )) {
/* If argument is a dependency/slice, add dependency to task dependency list */
Dependency *dep = dependency_new(arg_ptr, arg_size, arg_direction, arg_locality, task, accumulator, gatherv, task_args_list_node_ptr);
icl_list_t *task_dependency_list_node_ptr = icl_list_append( task->dependency_list, dep );
dep->task_dependency_list_node_ptr = task_dependency_list_node_ptr;
task->num_dependencies++;
task->num_dependencies_remaining++;
}
else if( arg_direction==SCRATCH ) {
Scratch *scratch = scratch_new( arg_ptr, arg_size, task_args_list_node_ptr);
icl_list_append( task->scratch_list, scratch );
}
}

Here is the call graph for this function:

Here is the caller graph for this function:

void QUARK_Waitall ( Quark *  quark)

Called by the master thread. Wait for all the tasks to be completed, then return. The worker tasks will also exit from their work loop at this time.

Parameters:
[in,out]quarkThe scheduler's main data structure.

Definition at line 790 of file quark.c.

References worker_s::finalize, quark_s::num_threads, QUARK_Barrier(), TRUE, and quark_s::worker.

{
int i;
QUARK_Barrier( quark );
/* Tell each worker to exit the work_loop; master handles himself */
for (i=1; i<quark->num_threads; i++)
quark->worker[i]->finalize = TRUE;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void QUARK_Worker_Loop ( Quark *  quark,
int  thread_rank 
)

This function is called by a thread when it wants to start working. This is used in a system that does its own thread management, so each worker thread in that system must call this routine to get the worker to participate in computation.

Parameters:
[in,out]quarkThe main data structure.
[in]thread_rankThe rank of the thread.

Definition at line 1702 of file quark.c.

References pthread_self(), worker_s::thread_id, work_main_loop(), and quark_s::worker.

{
quark->worker[thread_rank]->thread_id = pthread_self();
work_main_loop( quark->worker[thread_rank] );
}

Here is the call graph for this function: