Task #41

Addition of User and Host monitoring Graphical Interface

Added by José Luis Vázquez Poletti over 2 years ago. Updated over 2 years ago.

Status:Closed Start date:2009-08-03
Priority:High Due date:
Assignee:Srinivasan Natarajan % Done:

100%
Category:- Spent time: 38.90 hours
Target version:-

Description

As stated in Development Roadmap.

History

#1
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 0 to 10

The graphical user interface is designed for user information(gwuser) three GtkTreeview's are required for displaying

  • User Details
  • Execution Manager Drivers
  • Transfer Manager Drivers
gwgui@gridway:~$ gwuser
UID NAME       JOBS RUN IDLE   IDENTITY                                
0   gwgui      1    0   0      
    EM MADs: ws (PID=22484), prews_nsh (PID=22512)
    TM MADs: gridftp (PID=22531), dummy (PID=22550)

#2
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 10 to 20

The graphical user interface is designed for Job history information(gwhistory) using two GtkTreeview's for

  • Job State Information
  • Job History Details

#3
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 20 to 30

The user information view is implemented using GtKTreeView

  • The state attributes for user information view are

    ("USER_ID","USERNAME","ACTIVE_JOBS","RUNNING_JOBS","IDLE","IDENTITY")

  • The state attributes for Driver view are

    ("MAD_ID","NAME","PID")

The output of grid command(gwuser) will be returned back in xml format which needs to be parsed to get the proper data.This part has been implemented using libxml parser

<?xml version="1.0" encoding="iso-8859-1"?>
<gridway xmlns="http://gridway.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://gridway.org gridway.xsd">
<gwuser>
  <USER USER_ID="0">
    <USERNAME>gwgui</USERNAME>
    <ACTIVE_JOBS>2</ACTIVE_JOBS>
    <RUNNING_JOBS>1</RUNNING_JOBS>
    <IDLE>0</IDLE>
    <IDENTITY></IDENTITY>
    <EM_MAD MAD_ID="0">
      <NAME>ws</NAME>
      <PID>22484</PID>
    </EM_MAD>
    <EM_MAD MAD_ID="1">
      <NAME>prews_nsh</NAME>
      <PID>22512</PID>
    </EM_MAD>
    <TM_MAD MAD_ID="0">
      <NAME>gridftp</NAME>
      <PID>22531</PID>
    </TM_MAD>
    <TM_MAD MAD_ID="1">
      <NAME>dummy</NAME>
      <PID>22550</PID>
    </TM_MAD>
  </USER>
</gwuser>
</gridway>

#4
Updated by Srinivasan Natarajan over 2 years ago

The job history state information view needs to render a radio button for the user to select a particular job id.This part has been implemented using cell renderer radio toggle. The toggled callback is handled to deselect the previous selected option and handle the new selection.

#5
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 30 to 40

The job history view is implemented using GtKTreeView

  • The state attributes for job history view are
    "HID","START_TIME","EXIT_TIME","PROLOG","WRAPPER","EPILOG","MIGRATION","REASON","QUEUE","HOST"
The output of grid command(gwhistory) will be returned back in xml format which needs to be parsed to get the proper data.This part has been implemented using libxml parser
 <?xml version="1.0" encoding="iso-8859-1"?>
<gridway xmlns="http://gridway.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://gridway.org gridway.xsd">
<gwhistory JOB_ID="3">
  <HISTORY HISTORY_ID="0">
    <HID>1</HID>
    <START_TIME>19:37:37</START_TIME>
    <EXIT_TIME>19:37:37</EXIT_TIME>
    <PROLOG>0:00:00</PROLOG>
    <WRAPPER>0:00:00</WRAPPER>
    <EPILOG>0:00:00</EPILOG>
    <MIGRATION>0:00:00</MIGRATION>
    <REASON>err </REASON>
    <QUEUE>all.q</QUEUE>
    <HOST>aquila.dacya.ucm.es/SGE</HOST>
  </HISTORY>
  <HISTORY HISTORY_ID="1">
    <HID>0</HID>
    <START_TIME>19:37:32</START_TIME>
    <EXIT_TIME>19:37:32</EXIT_TIME>
    <PROLOG>0:00:00</PROLOG>
    <WRAPPER>0:00:00</WRAPPER>
    <EPILOG>0:00:00</EPILOG>
    <MIGRATION>0:00:00</MIGRATION>
    <REASON>err </REASON>
    <QUEUE>default</QUEUE>
    <HOST>hydrus.dacya.ucm.es/PBS</HOST>
  </HISTORY>
  <HISTORY HISTORY_ID="2">
    <HID>1</HID>
    <START_TIME>18:37:32</START_TIME>
    <EXIT_TIME>18:37:32</EXIT_TIME>
    <PROLOG>0:00:00</PROLOG>
    <WRAPPER>0:00:00</WRAPPER>
    <EPILOG>0:00:00</EPILOG>
    <MIGRATION>0:00:00</MIGRATION>
    <REASON>err </REASON>
    <QUEUE>all.q</QUEUE>
    <HOST>aquila.dacya.ucm.es/SGE</HOST>
  </HISTORY>
  <HISTORY HISTORY_ID="3">
    <HID>0</HID>
    <START_TIME>18:37:27</START_TIME>
    <EXIT_TIME>18:37:27</EXIT_TIME>
    <PROLOG>0:00:00</PROLOG>
    <WRAPPER>0:00:00</WRAPPER>
    <EPILOG>0:00:00</EPILOG>
    <MIGRATION>0:00:00</MIGRATION>
    <REASON>err </REASON>
    <QUEUE>default</QUEUE>
    <HOST>hydrus.dacya.ucm.es/PBS</HOST>
  </HISTORY>
</gwhistory>
</gridway>

#6
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 40 to 50

The graphical user interface is designed for host matching requirements of the job (gwhost -m) using two GtkTreeview's for displaying

  • Job Information
  • Hosts matching the requirements of a given job

#7
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 50 to 60

The hosts matching requirement of a job is implemented using GtKTreeView

  • The state attributes for the view are

    ("HID",QNAME","RANK","PRIO","SLOTS","HOSTNAME")

The output format below is parsed to load the matching requirement information in to memory.

<gwhost JOB_ID="3">
  <HOST HOST_ID="0">
    <HOSTNAME>hydrus.dacya.ucm.es</HOSTNAME>
    <FIXED_PRIORITY>1</FIXED_PRIORITY>
    <QUEUE QUEUE_ID="0">
      <QUEUE_NAME>default</QUEUE_NAME>
      <RANK>1995</RANK>
      <SLOTS>3</SLOTS>
    </QUEUE>
    <QUEUE QUEUE_ID="1">
      <QUEUE_NAME>qlong</QUEUE_NAME>
      <RANK>1995</RANK>
      <SLOTS>3</SLOTS>
    </QUEUE>
    <QUEUE QUEUE_ID="2">
      <QUEUE_NAME>verylong</QUEUE_NAME>
      <RANK>1995</RANK>
      <SLOTS>3</SLOTS>
    </QUEUE>
    <QUEUE QUEUE_ID="3">
      <QUEUE_NAME>qsmall</QUEUE_NAME>
      <RANK>1995</RANK>
      <SLOTS>3</SLOTS>
    </QUEUE>
  </HOST>
</gwhost>
</gridway>

#8
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 60 to 70
  • The gui for displaying all the host is implemented using GTKTreeview(all host view).
  • The user can select the host and get the queue information in host queue view.

The State attributes for the all host view has

    HOSTNAME
    FIXED_PRIORITY
    OS_NAME
    OS_VERSION
    ARCH
    CPU_MHZ
    CPU_FREE
    FREE_MEM_MB
    SIZE_MEM_MB
    FREE_DISK_MB
    SIZE_DISK_MB
    RUNNING_JOBS
    USED_SLOTS
    FREENODECOUNT
    NODECOUNT
    LRMS_NAME
    LRMS_TYPE

The state attributes for the host queue infomation has

    QUEUE_NAME
    QUEUE_FREENODECOUNT
    QUEUE_NODECOUNT
    QUEUE_MAXTIME
    QUEUE_MAXCPUTIME
    QUEUE_MAXCOUNT
    QUEUE_MAXRUNNINGJOBS
    QUEUE_MAXJOBSINQUEUE
    QUEUE_STATUS
    QUEUE_DISPATCHTYPE
    QUEUE_PRIORITY

#9
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 70 to 80

The output of grid command (gwhost) will be returned back in xml format which needs to be parsed to get the proper data.

The data from the remote grid system will have host information and queue information of all hosts.

The parsing must be done such that parser extracts only the host details and it will be loaded to the memory for all job state view.

This part has been implemented using libxml parser.

Below is the snapshot from the gwhost -x command

<?xml version="1.0" encoding="iso-8859-1"?>
<gridway xmlns="http://gridway.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://gridway.org gridway.xsd">
<gwhost>
  <HOST HOST_ID="0">
    <HOSTNAME>hydrus.dacya.ucm.es</HOSTNAME>
    <FIXED_PRIORITY>1</FIXED_PRIORITY>
    <OS_NAME>Linux</OS_NAME>
    <OS_VERSION>2.6.24-17-server</OS_VERSION>
    <ARCH>x86_64</ARCH>
    <CPU_MHZ>1995</CPU_MHZ>
    <CPU_FREE>100</CPU_FREE>
    <FREE_MEM_MB>1425</FREE_MEM_MB>
    <SIZE_MEM_MB>2014</SIZE_MEM_MB>
    <FREE_DISK_MB>5509</FREE_DISK_MB>
    <SIZE_DISK_MB>16362</SIZE_DISK_MB>
    <RUNNING_JOBS>0</RUNNING_JOBS>
    <USED_SLOTS>0</USED_SLOTS>
    <FREENODECOUNT>3</FREENODECOUNT>
    <NODECOUNT>4</NODECOUNT>
    <LRMS_NAME>PBS</LRMS_NAME>
    <LRMS_TYPE>pbs</LRMS_TYPE>
    <QUEUE QUEUE_ID="0">
      <QUEUE_NAME>default</QUEUE_NAME>
      <QUEUE_FREENODECOUNT>3</QUEUE_FREENODECOUNT>
      <QUEUE_NODECOUNT>4</QUEUE_NODECOUNT>
      <QUEUE_MAXTIME>0</QUEUE_MAXTIME>
      <QUEUE_MAXCPUTIME>-1</QUEUE_MAXCPUTIME>
      <QUEUE_MAXCOUNT>0</QUEUE_MAXCOUNT>
      <QUEUE_MAXRUNNINGJOBS>4</QUEUE_MAXRUNNINGJOBS>
      <QUEUE_MAXJOBSINQUEUE>0</QUEUE_MAXJOBSINQUEUE>
      <QUEUE_STATUS>enabled</QUEUE_STATUS>
      <QUEUE_DISPATCHTYPE>NULL</QUEUE_DISPATCHTYPE>
      <QUEUE_PRIORITY>0</QUEUE_PRIORITY>
    </QUEUE>
     ....
     ....
     ...
     ...
 </HOST>
</gwhost>
</gridway>

#10
Updated by Srinivasan Natarajan over 2 years ago

  • The output of grid command (gwhost) will be returned back in xml format which needs to be parsed to get the proper data.The data from the remote grid system will have host information and queue information of all hosts. The xml parsing for getting host information is already done
  • The parsing must be done such that parser extracts only queue details of a particular host and load it to the memory for GtkTreeview.
    This part has been implemented using libxml parser.
  • Refer the above snapshot from the gwhost -x command

#11
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 80 to 90

The output of gwacct will be returned from the remote grid system in xml format. The xml format will be different based on the
accounting statistics for the user/host.Changes are made to ensure proper parsing is done for the base tags(gwacct,USER,HOST) The secondary tags(TRANSFER,...) are common for both statistics.

Host Accounting snapshot:

<?xml version="1.0" encoding="iso-8859-1"?>
<gridway xmlns="http://gridway.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://gridway.org gridway.xsd">
<gwacct HOSTNAME="gridce2.pi.infn.it">
  <USER USERNAME="gwgui">
    <TRANSFER>00:00:00</TRANSFER>
    <EXECUTION>00:00:00</EXECUTION>
    <SUSPENSION>00:00:00</SUSPENSION>
    <TOTS>2</TOTS>
    <SUCC>0</SUCC>
    <ERR>2</ERR>
    <KILL>0</KILL>
    <USER>0</USER>
    <SUSP>0</SUSP>
    <DISC>0</DISC>
    <SELF>0</SELF>
    <PERF>0</PERF>
    <S-R>0</S-R>
  </USER>
</gwacct>
</gridway>

User Accounting snapshot:

<?xml version="1.0" encoding="iso-8859-1"?>
<gridway xmlns="http://gridway.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://gridway.org gridway.xsd">
<gwacct USERNAME="gwgui">
  <HOST HOSTNAME="agh2.atlas.unimelb.edu.a">
    <TRANSFER>00:00:00</TRANSFER>
    <EXECUTION>00:00:00</EXECUTION>
    <SUSPENSION>00:00:00</SUSPENSION>
    <TOTS>12</TOTS>
    <SUCC>0</SUCC>
    <ERR>12</ERR>
    <KILL>0</KILL>
    <USER>0</USER>
    <SUSP>0</SUSP>
    <DISC>0</DISC>
    <SELF>0</SELF>
    <PERF>0</PERF>
    <S-R>0</S-R>
  </HOST>
</gwacct>
</gridway>

#12
Updated by Srinivasan Natarajan over 2 years ago

The Xml contents in the memory needs to be converted in to a model for GtKTreeview.

This needs to be done for

  • Three user details view(gwuser)
  • Two accounting views(gwacct)
  • Four host information views(gwhost)
  • Two history views(gwhistory)

#13
Updated by Srinivasan Natarajan over 2 years ago

  • % Done changed from 90 to 100

Graphical user interface is designed for accounting information (gwacct) using two GtkTreeview's for displaying

  • Host Information
  • Statistics results.

The user can filter the results using date filter and can use user id/ hostname for the statistics

#14
Updated by José Luis Vázquez Poletti over 2 years ago

  • Status changed from Assigned to Closed

Also available in: Atom PDF