User Interfaces: Difference between revisions

From AstroEdWiki
Jump to navigation Jump to search
No edit summary
 
(37 intermediate revisions by the same user not shown)
Line 1: Line 1:
As part of our short course on [http://prancer.physics.louisville.edu/astrowiki/index.php/Python_for_Physics_and_Astronomy Python for Physics and Astronomy] we consider how users interact with their computing environment. A programming language such as Python provides tools to build code that computes scientific models, captures data, sorts it and analyzes it largely without operator action.  In effect, once you have written the program, you point it at the data or task it is to do, and wait for it to return new science to you.  This is the command line, or batch, model of computing and is at the core of large data science today. Indeed, from your handheld devices to supercomputers, the work that is done is for the most part autonomous.  We have seen how Python has built-in components to accept input from the command line, the operating system, the computer that is hosting the program, and the Internet or cloud.  What about the other side, the user's perspective on computing?
As an end user, would you prefer to move a mouse or tap a screen in order to select a file, or to type in the path and file name?  What if you had to make operational decisions based on graphical output, or changing real world environments as data are collected?  In modern computing, most of us interact with the machine and software through a graphical user interface or GUI.  These tools create that option.
'''On-Line Guides'''
*[http://www.tkdocs.com/ Tk]
*[https://matplotlib.org/users/index.html Matplotlib]
*[https://bokeh.pydata.org/en/latest/docs/reference.html#refguide Bokeh]


As part of our short course on [http://prancer.physics.louisville.edu/astrowiki/index.php/Python_for_Physics_and_Astronomy Python for Physics and Astronomy] we consider how users interact with their computing environment. A programming language such as Python provides tools to build code that computes scientific models, captures data, sorts it and analyzes it largely without operator action.  In effect, once you have written the program, you point it at the data or task it is to do, and wait for it to return new science to you.  This is the command line, or batch, model of computing and is at the core of large data science today. Indeed, from your handheld devices to supercomputers, the work that is done is for the most part autonomous. We have seen how Python has built-in components to accept input from the command line, the operating system, the computer that is hosting the program, and the Internet or cloudWhat about the other side, the user's perspective on computing?
While the conventional Tk and Matplotlib components are foundational to Python, Bokeh is a very recent development with the design philosophy to put the web first for the end user and it has a  contemporary look. It also enables adding widgets written for javascript within the web display, which can be be very effective.   


As an end user, would you prefer to move a mouse or tap a screen in order to select a file, or to type in the path and file name?  What if you had to make operational decisions based on graphical output, or changing real world environments as data are collected?  In modern computing, most of us interact with the machine and software through a graphical user interface or GUI. 




== Command Line Interfacing and Access to the  Operating System ==
=== Command Line Interfacing and Access to the  Operating System ===




In a unix-like enviroment (Linux or MacOS), the command line is an accessible and often preferred way to instruct a program on what to do. A typical program, as we've seen, might start like this example to interpolate a data  file and plot the result:
In a Unix-like enviroment (Linux or MacOSX), the command line is an accessible and often preferred way to instruct a program on what to do. A typical program, as we've seen, might start like this example to interpolate a data  file and plot the result:


   #!/usr/bin/python
   #!/usr/bin/python
Line 138: Line 146:
   #!/usr/bin/python
   #!/usr/bin/python


   # Process images in a directory  
   # Process images in a directory tree


   import os
   import os
Line 151: Line 159:
   sys.exit("Usage: process_fits.py directory\n")
   sys.exit("Usage: process_fits.py directory\n")


   toplevel = sys.argv[-1]
   toplevel = sys.argv[1]


   # Search for files with this extension
   # Search for files with this extension
Line 182: Line 190:




=== Graphical User Interface to Plotting ===


== Graphical User Interfacing ==
First, read the  [http://prancer.physics.louisville.edu/astrowiki/index.php/Graphical_User_Interface_with_Python comprehensive section on Tkinter]  to see how that code works, and then the one on [http://prancer.physics.louisville.edu/astrowiki/index.php/Graphics_with_Python graphics with Python] to learn the basics of the plotting toolkits.  In this section we combine Tk for control with  interactive graphics.  Our goals are to


How do we get to "point-and-click" operations that take us beyond the packaged pyplot routines to code of our own?  Python offers a built-in package that is easy to use (for simple applications) and adapts to the operating system so that the programs have the look and feel of the native OS.  It also has add-on modules that may be installed on some systems to make GUI interfaces in the style of Gnome (GTK) or KDE (Qt) as well as others.  We will focus on the built-in "Tk" library and its use because it is simple and effective, and it may satisfy most needs without adding complexity that we usually associate with other programming languages like C++, or native C. 
* Retain the features of the graphics display with its interactivity and style
* Use tkinter to offer the user access to new features such  loading files and processing data
* Allow real-time updating so that the plot can follow changing data


[https://wiki.python.org/moin/TkInter Tkinter] is Python's standard GUI that is built on a library called [http://www.tcl.tk/ Tck/Tk] that is available on all operating systems. There are layers of software here, starting at the low-level of the operating system, and finishing with the top level of the user's experience.  Python connects these with Tkinter, or just Tk for short.  If you [https://www.google.com/?gws_rd=ssl#q=python+tk Google Python Tk] you will see many links to tutorials and reference documentation. Take care in following that journey. The path through them is difficult and and you may not return soon!
To this end we will write a Python 3 program that uses tkinter and add matplotlib or bokeh to make useful tools that also serve as templates of your own development. The two resulting programs are almost identical except for the plotting functions, and you will find them on the [http://prancer.physics.louisville.edu/classes/650/python/examples/ examples page]. Look for "tk_plot.py" and "bokeh_plot.py".


A simpler alternative is to follow a few examples here that may be enough to get you started, and then go to Google when you are stuck with your particular applicationI have excerpted these from [http://www.rmi.net/~lutz/about-pp.html Programming Python by Mark Lutz (O'Reilly 2011)] which is highly recommended for self-study.  Programming Python has been updated to include Python 3 too.  Here's a 4-line program:
Before we begin, check that bokeh and tkinter are available in your version of Python 3The version of Tk should be at least 8.6, which you can check with


  from Tkinter import *
  tkinter.TkVersion
  widget = Label(None, text='Hello world!')
  widget.pack()
  widget.mainloop()


That first line that imports tkinter is written here for Python 2.7.  If you use Python 3, it will be "tkinter" insteadOtherwise, it should work similarly in both systems. The import * brings in all the components and will let you use them without a prefix like "Tkinter.", which should not be a problem for most applications because tk is a well-established part of Python.  The second line creates a "Label" widget and puts text into it.  The third places widget in the window, and the fourth starts and runs the GUI. That's it.  Once it is running you will see something that looks like this:
on the command line after importing tkinter.  For bokeh, use


  bokeh.__version__


[[File:Tk_hello.png | center]]
that's with two underscores before and after the "version".  Look for version 0.12.15 or greater to have the functionality described here.


Tk handles the window and the usual operations with it.  The Label  widget has several options, and entering them when it is created in this way is the most direct.  You could also do it like this:


  from Tkinter import *
'''The Tk Framework'''
  widget = Label()
  widget['text'] = 'Hello world!'
  widget.pack(side=TOP)
  widget.mainloop()


where we have changed the widget's content with a mapping key, and in this example, told "pack" where to put it.  We will see more examples of the packing operation later.
We begin our code as usual by requiring these libraries




Within the mainloop (the details of which we do not see) the sytem is waiting for the user to do something.  It reacts to your action and then goes back to waiting.  Most of the time it is not doing anything except waiting for you.  If you want the system to be doing something else, and then checking on you too, you have to cautiously program the actions of the GUI so that it remains responsive, or else spawn another process to handle other those tasks while the GUI waits on you.  In those cases, the programming problem is to establish communication with the processes which would be running asynchronously to the main loop.  We'll leave that one for you to ponder on a rainy night. 
  import tkinter as tk
  from tkinter import ttk
  from tkinter import filedialog
  from tkinter import messagebox


Now the main loop is running and waiting, but you have not asked it to do anything. It is just waiting for one of the available system window operations like resize or kill. We can add buttons which respond when clicked, and then run a "callback" routineThat is, you click the button, Tk captures the click and tells your program to run a specific routine because you hit that button.  Routines that respond in this way are called ''callbacks''.
such that Tk functions require the "tk." and ttk functions use "ttk".  We have also included file dialog  and message widgets that were mentioned in the summary of Tk widgets.


For connection to the operating system we need "os" and "sys", and for handling data we use numpy


  import os
   import sys
   import sys
   from Tkinter import *
   import numpy as np
 
 
   lastwords='You can start this program again and I will come back.'
There are global variables that are used to pass information from file handlers and processing to the graphics components
   def saygoodbye():
 
     print('Well, if you are  going to act that way I am leaving.')
  global selected_files
     print lastwords
  global x_data
     sys.exit()
  global y_data
 
  selected_files = []
  x_data = np.zeros(1024)
  y_data = np.zeros(1024)
  x_axis_label = ""
   y_axis_label = ""
 
We will create a Tk window with button or other widgets that require call backs when they are activated.  Since these programs are templates for what can be done, look at the examples to see how the call backs are structured.  The one to read a data file illustrates how to use Python to parse a file and save its data in numpy arrays.
 
   def read_file(infile):
     global x_data
    global y_data
 
    datafp = open(infile, 'r')
    datatext = datafp.readlines()
    datafp.close()
 
    # How many lines were there?
 
    i = 0
    for line in datatext:
      i = i + 1
 
    nlines = i
 
    # Fill  the arrays for fixed size is much faster than appending on the fly
 
    x_data = np.zeros((nlines))
    y_data = np.zeros((nlines))
 
    # Parse the lines into the data
 
    i = 0
    for line in datatext:
     
      # Test for a comment line
     
      if (line[0] == "#"):
        pass
       
      # Treat the case of a plain text comma separated entries   
     
      try:
             
        entry = line.strip().split(",") 
        x_data[i] = float(entry[0])
        y_data[i] = float(entry[1])
 
        i = i + 1   
      except:     
     
        # Treat the case of space separated entries
     
        try:
          entry = line.strip().split()
          x_data[i] = float(entry[0])
          y_data[i] = float(entry[1])
          i = i + 1
        except:
          pass
 
    return()
 
Notice how we allow for both comma separated and space delimited data.  The expectation is that the file will have two values per line, the first one being "x" and the second one being "y".  They may have white space between them, or be separated by a comma.  Files written this way are very common, and easy to use too, but we may not know before reading one which style it was written in.  Also common (in Grace, for example), a "#" at the beginning of  a line indicates a comment and implies to ignore the entire line.  The reader simply skips lines that begin with "#". A more advanced reader would validate the numbers as they come in to prevent errors later. This one simply assigns them to two global arrays, one for x and one for y, because that is the format required for plotting 2D data by both matplotlib and bokeh. Also, having the data in numpy offers the options of other processing based on the GUI.
 
The file that is being read has been selected with a Tk widget that returns filenames in a global list
 
  def select_file():
 
    global selected_files
   
    # Use the tk file dialog to identify file(s)
   
    newfile = ""
    try:
      newfile, = tk.filedialog.askopenfilenames()
      selected_files.append(newfile)
     except:
      tk_info.set("No file selected")
 
     if newfile !="":
      tk_info.set("Latest file: "+newfile)
 
    return()
 
By holding onto all the selections in s list, we retain the option of going back to them later.  However here in the file selection call back, we take only the first file that the user selects to add to that list. Of course we take all of them and process the one by one.  The Tk function will return leaving the selected_files list with its new entry as the last one on the list, and display its name on the user interface.
 
 
 
'''Matplotlib from Tk on the Desktop'''
 


For matplotlib we need


  widget = Button(None, text='Goodbye world!', command=saygoodbye)
import matplotlib as mpl
  widget.pack()
import matplotlib.pyplot as plt
  widget.mainloop()
mpl.use('TkAgg')


The Plot button call back uses matplotlib with its pyplot namespace to create a plot on the matplotlib canvas.  The plot is not embedded in the Tk user interface in order to invoke the matplotlib toolbar, which in version 2.2 is deprecated for Tk.  This solution avoids that issue, but also means that it is not possible to update the content of the displayed data through the Tk interface.


This one makes a button instead of a label.  Click the button and the program exits, writing on the screen (not in the GUI) the print statement's text. The callback to exit here is simple and does not take an argument.  In this example we have passed information to the callback with a global variable ''lastwords''.  How to manage arguments and global variables within callbacks is a broader problem and diverts us from the task at hand of seeing how to build a GUI.  Just keep in mind, that if you have to send a value or parameters in a callback, there are other tricks to learn, including different schemes of specifying the callback that utilize the object oriented features of Python.
  # Create the desired plot
 
  def make_plot(event=None):
   
    global selected_files
    global x_axis_label
    global y_axis_label
   
    nfiles = len(selected_files)
    this_file = selected_files[nfiles-1]
   
    read_file(this_file)


    # Create the plot using bokeh
   
    this_file_basename = os.path.basename(this_file)
    base, ext = os.path.splitext(this_file_basename)
    bokeh_file = base+".html"
   
    output_file(bokeh_file)
    p = figure(tools="hover,crosshair,pan,wheel_zoom,box_zoom,box_select,reset")
    p.line(x_data, y_data, line_width=2)
    show(p)
  # Create the desired plot with matplotlib


We will take this one more level and put five widgets at once by using the concept of a frame and a root window in this example adapted from [http://www.tutorialspoint.com/python/tk_frame.htm  Python tutorials]:
  def make_plot(event=None):
   
    global selected_files
    global x_axis_label
    global y_axis_label
   
    nfiles = len(selected_files)
    this_file = selected_files[nfiles-1]
   
    read_file(this_file)


    # Create the plot.
    plt.figure(nfiles)
    plt.plot(x_data, y_data, lw=3)
    plt.title(this_file)
    plt.xlabel(x_axis_label)
    plt.ylabel(y_axis_label)
    plt.show()


  import sys
  from Tkinter import *


Input is handled through global variables, and the axis labels may be assigned through the Tk interface, though in tk_plot.py that is left for the next version. 


  lastwords='You can start this program again and I will come back.'


  def saygoodbye():
[[File:Humidity_tk.png]]
    print('Well, if you are  going to act that way I am leaving.')
    print lastwords
    sys.exit()




  root = Tk()
  topframe = Frame(root)
  frame.pack( side = TOP )


  bottomframe = Frame(root)
  bottomframe.pack( side = BOTTOM )


  redbutton = Button(topframe, text="Red", fg="red")
  redbutton.pack( side = LEFT)


  greenbutton = Button(topframe, text="Brown", fg="brown")
'''Bokeh from Tk in the Browser and on the Web'''
  greenbutton.pack( side = LEFT )


  bluebutton = Button(topframe, text="Blue", fg="blue")
We include the bokeh modules needed for a basic plot
  bluebutton.pack( side = LEFT )


   blackbutton = Button(bottomframe, text="Black", fg="black")
   from bokeh.plotting import figure, output_file, show
  blackbutton.pack( side = LEFT)


  byebutton = Button(bottomframe, text="Exit", fg="magenta", command=saygoodbye )
For bokeh the call back is very similar
  byebutton.pack( side = RIGHT)


   root.mainloop()
   # Create the desired plot with bokeh


  def make_plot(event=None):
   
    global selected_files
    global x_axis_label
    global y_axis_label
   
    nfiles = len(selected_files)
    this_file = selected_files[nfiles-1]
   
    read_file(this_file)


Notice we had to have a root window in which to place the frames, and that we made two frames in this window. The buttons in usual operation would each have a callback that would do something other than simply exit. The placement of the buttons was handled by pack(), selecting in the program preferences that pack manages for us. If we need more control, then we can add an anchor keyword to the pack statement. Anchor has arguments "N,S,E,W,NE,NW,SE,SW,CENTER) which tells pack where to put the widget within its allowed space. There are other pack options too, such as expand and fill, which can be used to customize the appearance of the GUI.
    # Create the plot using bokeh
   
    this_file_basename = os.path.basename(this_file)
    base, ext = os.path.splitext(this_file_basename)
    bokeh_file = base+".html"
   
    output_file(bokeh_file)
    p = figure(tools="hover,crosshair,pan,wheel_zoom,box_zoom,box_select,reset")  
    p.line(x_data, y_data, line_width=2)
    show(p)


Tkinter has a wide selection of widget classes that will handle most common GUI applications:


*Label        Message area
The tools are explicitly requested, unlike matplotlib which provides a tool bar that is fully populated.
*Button      Push-button
*Frame        Container for other widgets
*Toplevel,Tk  A window managed by the system window manager
*Message      Multiline label
*Entry        Single line of text entry
*Checkbutton  Two-state multiple choice button
*Radiobutton  Two-state single choice button
*Scale        Slider for changing scales
*PhotoImage  Full color image object
*BitmapImage  Bitmap image
*Menu        Set of options
*Scrollbar    Scrolls other widgets, such as a canvas
*Listbox      Selection of names
*Text        Multiline text for editing
*Canvas      Graphic drawing area
*FileDialog  Open or get the name of a file


[[File:Humidity_bokeh.png]]


Opening a file is fundamental to many uses of Python.  We have seen that matplotlib has useful interactive graphics in a GUI, so how do we incorporate that into a root window of our own, and add file selection?




Here is  a data plotting program that illustrates many of the key elements of Tk GUI programming.  The complete program is available [http://prancer.physics.louisville.edu/classes/650/python/examples  here] as pyplot_data.example.  Let's look at how this works.




First, we tell the operating system to use Python, and we comment the code with a simple description of what it does.
=== Running a Bokeh Server for  Live Plotting of Python Data ===


  #!/usr/bin/python
Lastly, we arrive at the destination:  a solution to interactive plotting where data are created and modified in Python, and presented to the user on the fly, with a customized and responsive interface.  The interface components can be entirely in the browser, and thereby potentially offered to a web client, or they can be shared between the browser and a Python GUI, for desktop applications.  There are three components:
  """ Interactively plot data using matplotlib pyplot within a Tk root window
    """


We import sys and numpy modules and use numpy as "np" in the usual way.
* Python backend, perhaps with Tk or other GUI or responding to CGi requests from another server
* Bokeh server responding to the Python and presenting information to the browser
* Browser client, listening to the Bokeh server directly or through a proxy, and providing data to the Python backend if needed


  import sys   
For details on how to set this up and  its possible uses, see
  import numpy as np


We import matplotlib and call it mpl for a short name. We tell mpl to use Tk by default.
[https://bokeh.pydata.org/en/latest/docs/user_guide/server.html Running a Bokeh Server]


  import matplotlib as mpl
Several examples are offered here
  mpl.use('TkAgg')


We import some backend things that may (or may not) be needed to get plotting to work in our own window.
[https://demo.bokehplots.com/ demo.bokehplots.com]


from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg
The first live example shown there is from "sliders.py", a copy of which is in our [http://prancer.physics.louisville.edu/classes/650/python/examples examples directory].
from matplotlib.backend_bases import key_press_handler


We import Figure which does does the plotting for us


from matplotlib.figure import Figure
  # Load the modules


Importing Tk is handled differently in Python 2 and Python 3. We also get askopenfilename which we will use to get a file using a GUI.
  import numpy as np


if sys.version_info[0] < 3:
  from bokeh.io import curdoc
   import Tkinter as Tk
   from bokeh.layouts import row, widgetbox
   from tkFileDialog import askopenfilename
   from bokeh.models import ColumnDataSource
else:
   from bokeh.models.widgets import Slider, TextInput
   import tkinter as Tk
   from bokeh.plotting import figure
   from tkFileDialog import askopenfilename


This program optionally will take a filename on the command line.  We define a flag to tell us later if there was one provided when the program started.
  # Set up data using numpy


   fileflag = True
   N = 200
  x = np.linspace(0, 4*np.pi, N)
  y = np.sin(x)


Parse the command line and set the fileflag according to the user's wishes.
  # Set up the display data source


   if len(sys.argv) == 1:
   source = ColumnDataSource(data=dict(x=x, y=y))
    fileflag = False
  elif len(sys.argv) == 2:
    fileflag = True
    infile = sys.argv[1]
  else:
    print " "
    print "Usage: pyplot_data.py [indata.dat]"
    print " "
    sys.exit("\nUse pyplot to display a data file\n")


If there was no file provided, bring up a GUI to allow the user to select a file.
  # Set up plot


   if not fileflag:
   plot = figure(plot_height=400, plot_width=400, title="my sine wave",
    root = Tk.Tk()
  tools="crosshair,pan,reset,save,wheel_zoom",
    infile = askopenfilename()
  x_range=[0, 4*np.pi], y_range=[-2.5, 2.5])
    root.quit()
    root.destroy()  


After the file selection has been done, we need to remove traces of the is window so it will not clutter the desktop or cause errors later.
  plot.line('x', 'y', source=source, line_width=3, line_alpha=0.6)


Open the file with the data if we can, and if not exit gracefully.
  # Set up widgets similar to Tk


   try:
   text = TextInput(title="title", value='my sine wave')
    infp = open(infile, 'r')
  offset = Slider(title="offset", value=0.0, start=-5.0, end=5.0, step=0.1)
 
   amplitude = Slider(title="amplitude", value=1.0, start=-5.0, end=5.0, step=0.1)
   except:
  phase = Slider(title="phase", value=0.0, start=0.0, end=2*np.pi)
    sys.exit("File %s could not be opened\n" % (infile,))


Read all the lines from the data into a list of text lines.
  # Add interactive tools


   intext = infp.readlines()
   freq = Slider(title="frequency", value=1.0, start=0.1, end=5.1, step=0.1)


Split data text and parse into x,y values. Start by createing  empty lists for the data.  We need to use lists since numpy arrays are immutable and we do not know how much data we have yet.
  # Set up callbacks to the widgets


   xdata = []
   def update_title(attrname, old, new):
  ydata = []
      plot.title.text = text.value
  i = 0


   for line in intext:
   text.on_change('value', update_title)


    try:
  def update_data(attrname, old, new):
      # Treat the case of a plain text comma separated entry
   
      entry = line.strip().split(",")  


       # Get the x,y values for these fields
       # Get the current slider values
       xval = float(entry[0])
       a = amplitude.value
       yval = float(entry[1])
       b = offset.value
      xdata.append(xval)
       w = phase.value
       ydata.append(yval)
       k = freq.value
       i = i + 1   


    except:     
      # Generate the new curve
   
      x = np.linspace(0, 4*np.pi, N)
       try:
       y = a*np.sin(k*x + w) + b
        # Treat the case of a plane text blank space separated entry


        entry = line.strip().split()
      source.data = dict(x=x, y=y)


        xval = float(entry[0])
  for w in [offset, amplitude, phase, freq]:
        yval = float(entry[1])
      w.on_change('value', update_data)
        xdata.append(xval)
        ydata.append(yval)
        i = i + 1   
       
      except:
        pass   
     
  # How many points found?


  nin = i
  if nin < 1:
    sys.exit('No objects found in %s\n' % (infile,))


In scanning the text lines we look for x,y pairs separated by whitespace, or by a comma. They have to be parsed differently, so we try each format.  When we are done we note how many lines were successfully read.
  # Set up layouts and add to document
  inputs = widgetbox(text, offset, amplitude, phase, freq)


Now that the data are fixed, we import x and y into numpy arrays.
  curdoc().add_root(row(inputs, plot, width=800))
  curdoc().title = "Sliders"


  x = np.array(xdata)
  y = np.array(ydata)


The data are now loaded and ready to plot, so the rest is GUI content.
Download the file sliders.py or copy the source shown, and on a computer that has Python 3 and Bokeh installed, use the command line


Create the root/toplevel window with title
  bokeh server sliders.py


  root = Tk.Tk()
to initiate a live session in the server. Once that has started, open your browser to "localhost", that is to your own computer, by entering this on the browser source line
  root.wm_title("PyPlot")


  http://localhost:5006/sliders


# Use this for HD sized display
The display will look like this, except the sliders will cause changes in the plot.
#f = Figure(figsize=(16,9), dpi=100)


# Use this for smaller sized displays


Set up the figure for the plot.  Here we use a 7x5 aspect ratio at 200 DPI.  On the screen his is nominally a 7x5 inch display, but depends on the monitor.  Try 16x9 with 100 DPI for a larger monitor.  Add a title to the plot and label the axes.
[[File:Sliders.png]]


  f = Figure(figsize=(7,5), dpi=200)
  a = f.add_subplot(111)
  a.plot(x,y)
  a.set_title(infile)
  a.set_xlabel('X')
  a.set_ylabel('Y')


Create tk.DrawingArea with mpl figure


  canvas = FigureCanvasTkAgg(f, master=root)
  canvas.show()
  canvas.get_tk_widget().pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)


Optionally add the default mpl plotting toolbar (comment out to disable)
=== Running a Server for Javascript in a Browser Engine ===


  toolbar = NavigationToolbar2TkAgg( canvas, root )
Python includes packages that enable a simple webserver which may be used to run advanced graphics operations through javascript within a browser's javascript engine.  We will cover use of javascript, and Three.js in particular, as a supplement or replacement for 3D visualization in Python.  In order to do this without the burden of managing a full Apache installation, we turn to Python. This shell script in Linux will start a web server in the directory that the script is running in:
  toolbar.update()


Pack the canvas to make things fit nicely.
  python -m CGIHTTPServer 8000 1>/dev/null 2>/dev/null &
  echo "Use localhost:8000"
  echo


  canvas._tkcanvas.pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)
By using port 8000 the server is distinct from the one on port 80 used for web applications. The site would appear by putting


Add the keypress event handler that is built into mpl.  This may not be necessary, but it seems wise.
  http://localhost:8000


  def on_key_event(event):
in a Google Chrome or Mozilla Firefox browser window running on the same user account on the same machine. Note the redirects for stdio and stderr to /dev/null keeps output from appearing in the console.  The server may be killed by identifying its process ID in Linux with the command
    print('You pressed %s'%event.key)
    key_press_handler(event, canvas, toolbar)


   canvas.mpl_connect('key_press_event', on_key_event)
   ps -e | grep python


Add a Quit button as an example of what you might include on this canvas.
followed by


   def mpl_quit():
   kill -s 9  pid


    # Stop mainloop
where "pid" is the  ID number found in the first lineAlternatively, if it is the only python process running you may kill it with
    root.quit()
 
    # Prevent error running on Windows OS
    root.destroy()  
 
  button = Tk.Button(master=root, text='Quit', command=mpl_quit)
  button.pack(side=Tk.BOTTOM)


Now we are ready to run the loop interact with the graphics.
  killall python


   Tk.mainloop()
Any file in the directory tree below the starting directory is now accessible in the browser, and html files will be parsed to run the included javascript.  If here is a cgi-bin directory at the top level, the server will see it and use it. One use of this low level server is to create a virtual instrument that is accessible from the web, but not exposed to it directly.  A remote web server on the same network that can access port 8000 on the instrument machine can run code and get response from the  instrument by calling cgi-bin operations.    


The program runs waiting for user input.  When you move the mouse, those events are trapped by mpl and used to update the cursor readout, or to activate the buttons on the mpl toolbar.  When you click your Exit button the root window will be removed and the program will exit.
For programmers, however, this utility allows development and debugging of web software without the need for a large server.

Latest revision as of 07:02, 17 April 2018

As part of our short course on Python for Physics and Astronomy we consider how users interact with their computing environment. A programming language such as Python provides tools to build code that computes scientific models, captures data, sorts it and analyzes it largely without operator action. In effect, once you have written the program, you point it at the data or task it is to do, and wait for it to return new science to you. This is the command line, or batch, model of computing and is at the core of large data science today. Indeed, from your handheld devices to supercomputers, the work that is done is for the most part autonomous. We have seen how Python has built-in components to accept input from the command line, the operating system, the computer that is hosting the program, and the Internet or cloud. What about the other side, the user's perspective on computing?

As an end user, would you prefer to move a mouse or tap a screen in order to select a file, or to type in the path and file name? What if you had to make operational decisions based on graphical output, or changing real world environments as data are collected? In modern computing, most of us interact with the machine and software through a graphical user interface or GUI. These tools create that option.

On-Line Guides

While the conventional Tk and Matplotlib components are foundational to Python, Bokeh is a very recent development with the design philosophy to put the web first for the end user and it has a contemporary look. It also enables adding widgets written for javascript within the web display, which can be be very effective.


Command Line Interfacing and Access to the Operating System

In a Unix-like enviroment (Linux or MacOSX), the command line is an accessible and often preferred way to instruct a program on what to do. A typical program, as we've seen, might start like this example to interpolate a data file and plot the result:

 #!/usr/bin/python
 import sys
 import numpy as np
 from scipy.interpolate import UnivariateSpline
 import matplotlib.pyplot as plt
 sfactorflag = True
 if len(sys.argv) == 1:
   print " "
   print "Usage: interpolate_data.py indata.dat outdata.dat nout [sfactor]"
   print " "
   sys.exit("Interpolate data with a univariate spline\n")
 elif len(sys.argv) == 4:
   infile = sys.argv[1]
   outfile = sys.argv[2]
   nout = int(sys.argv[3])
   sfactorflag = False
 elif len(sys.argv) == 5:
   infile = sys.argv[1]
   outfile = sys.argv[2]
   nout = int(sys.argv[3])
   sfactor = float(sys.argv[4]) 
 else:
   print " "
   print "Usage: interpolate_data.py indata.dat outdata.dat nout [sfactor]"
   print " "
   sys.exit("Interpolate data with a univariate spline\n")
 

It uses "sys" to parse the command line arguments into text and numbers that control what the program will do. Because its first line directs the system to use the python interpreter, if the program is marked as executable to the user it will run as a single command followed by arguments. In this case it would be something like

 interpolate_data.py indata.dat outdata.dat nout sfactor

where indata.dat is a text-based data file of x,y pairs, one pair per line, outdata.dat is the interpolated file, nout is the number of points to be interpolated, and sfactor is an optional floating point smoothing factor. When you run this it will read the files, do the interpolation without further interaction, and (as written) plot a result as well as write out a data file. The rest of the code is

 # Take x,y coordinates from a plain text file
 # Open the file with data
 infp = open(infile, 'r')
 # Read all the lines into a list
 intext = infp.readlines()
 # Split data text and parse into x,y values  
 # Create empty lists
 xdata = []
 ydata = []
 i = 0  
 for line in intext:  
   try:
     # Treat the case of a plain text comma separated entry   
     entry = line.strip().split(",") 
     # Get the x,y values for these fields
     xval = float(entry[0])
     yval = float(entry[1])
     xdata.append(xval)
     ydata.append(yval)
     i = i + 1    
   except:          
     try: 
       # Treat the case of a plane text blank space separated entry
       entry = line.strip().split()
       xval = float(entry[0])
       yval = float(entry[1])
       xdata.append(xval)
       ydata.append(yval)
       i = i + 1             
     except:
       pass     
 # How many points found?  
 nin = i
 if nin < 1:
   sys.exit('No objects found in %s' % (infile,))
 
 # Import  data into a np arrays  
 x = np.array(xdata)
 y = np.array(ydata)


 # Function to interpolate the data with a univariate cubic spline
 if sfactorflag:
   f_interpolated = UnivariateSpline(x, y, k=3, s=sfactor)
 else:
   f_interpolated = UnivariateSpline(x, y, k=3)


 # Values of x for sampling inside the boundaries of the original data
 x_interpolated = np.linspace(x.min(),x.max(), nout)
 # New values of y for these sample points
 y_interpolated = f_interpolated(x_interpolated)


 # Create an plot with labeled axes
 plt.figure().canvas.set_window_title(infile)
 plt.xlabel('X')
 plt.ylabel('Y')
 plt.title('Interpolation')
 plt.plot(x, y,   color='red', linestyle='None', marker='.', markersize=10., label='Data')
 plt.plot(x_interpolated, y_interpolated, color='blue', linestyle='-', marker='None', label='Interpolated', linewidth=1.5)
 plt.legend()
 plt.minorticks_on()
 plt.show()
 
 # Open the output file
 outfp = open(outfile, 'w')
 # Write the interpolated data
 for i in range(nout):   
   outline = "%f  %f\n" % (x[i],y[i])
   outfp.write(outline)
 # Close the output file
 outfp.close()
 
 # Exit gracefully
 exit()


Aftet the fitting is done the program runs pyplot to display the results. The interactive window it opens and manages is a GUI, but it has been set up by the command line code. Of course there are many variations on command line interfacing, and the one shown here with coded argument parsing is perhaps the simplest and would serve as a template for most applications. Python offers other ways to manage the command line too. The os module is useful to have access to the operating system from within a Python routine. Some examples are

 import os
 os.chdir(path) changes the current working directory (CWD) to a new one
 os.getcdw() returns the CWD
 os.getenv(varname) returns the value of the environment variable varname

and there are many more, providing within the Python program many of the command line operating system tools available on the system. Here's an example of how that might be used in a program that processes many files in a directory:

 #!/usr/bin/python
 # Process images in a directory tree
 import os
 import sys
 import fnmatch
 import string
 import subprocess
 import pyfits
 if len(sys.argv) != 2:
  print " "
  sys.exit("Usage: process_fits.py directory\n")
 toplevel = sys.argv[1]
 # Search for files with this extension
 pattern = '*.fits'  
 for dirname, dirnames, filenames in os.walk(toplevel):
   for filename in fnmatch.filter(filenames, pattern):
     fullfilename = os.path.join(dirname, filename)
   
     try:    
   
       # Open a fits image file
       hdulist = pyfits.open(fullfilename)
     
     except IOError: 
       print 'Error opening ', fullfilename
       break       
     # Do the work on the files here ...
     
     # You can call a separate system process outside of Python this way
     darkfile = 'dark.fits'
     infilename = filename
     outfilename = os.path.splitext(os.path.basename(infilename))[0]+'_d.fits'
     subprocess.call(["/usr/local/bin/fits_dark.py", infilename, darkfile, outfilename]) 
 exit()

Here we used the os module's routines to walk through a directory tree, parse filenames, and then perform another operation on those files that is a separate command line Python program. Command line tools used to leverage the operating system's built-in functions can be very powerful, and take hours out of actually running a program on a large database.


Graphical User Interface to Plotting

First, read the comprehensive section on Tkinter to see how that code works, and then the one on graphics with Python to learn the basics of the plotting toolkits. In this section we combine Tk for control with interactive graphics. Our goals are to

  • Retain the features of the graphics display with its interactivity and style
  • Use tkinter to offer the user access to new features such loading files and processing data
  • Allow real-time updating so that the plot can follow changing data

To this end we will write a Python 3 program that uses tkinter and add matplotlib or bokeh to make useful tools that also serve as templates of your own development. The two resulting programs are almost identical except for the plotting functions, and you will find them on the examples page. Look for "tk_plot.py" and "bokeh_plot.py".

Before we begin, check that bokeh and tkinter are available in your version of Python 3. The version of Tk should be at least 8.6, which you can check with

  tkinter.TkVersion

on the command line after importing tkinter. For bokeh, use

 bokeh.__version__

that's with two underscores before and after the "version". Look for version 0.12.15 or greater to have the functionality described here.


The Tk Framework

We begin our code as usual by requiring these libraries


 import tkinter as tk
 from tkinter import ttk
 from tkinter import filedialog
 from tkinter import messagebox

such that Tk functions require the "tk." and ttk functions use "ttk". We have also included file dialog and message widgets that were mentioned in the summary of Tk widgets.

For connection to the operating system we need "os" and "sys", and for handling data we use numpy

 import os
 import sys
 import numpy as np

There are global variables that are used to pass information from file handlers and processing to the graphics components

 global selected_files
 global x_data
 global y_data
 selected_files = []
 x_data = np.zeros(1024)
 y_data = np.zeros(1024)
 x_axis_label = ""
 y_axis_label = ""

We will create a Tk window with button or other widgets that require call backs when they are activated. Since these programs are templates for what can be done, look at the examples to see how the call backs are structured. The one to read a data file illustrates how to use Python to parse a file and save its data in numpy arrays.

 def read_file(infile):
   global x_data
   global y_data
   datafp = open(infile, 'r')
   datatext = datafp.readlines()
   datafp.close()
   # How many lines were there?
   i = 0
   for line in datatext:
     i = i + 1
   nlines = i
   # Fill  the arrays for fixed size is much faster than appending on the fly
   x_data = np.zeros((nlines))
   y_data = np.zeros((nlines))
   # Parse the lines into the data
   i = 0
   for line in datatext:
     
     # Test for a comment line
     
     if (line[0] == "#"):
       pass
        
     # Treat the case of a plain text comma separated entries    
     
     try:
             
       entry = line.strip().split(",")  
       x_data[i] = float(entry[0])
       y_data[i] = float(entry[1])
       i = i + 1    
     except:      
     
       # Treat the case of space separated entries
     
       try:
         entry = line.strip().split()
         x_data[i] = float(entry[0])
         y_data[i] = float(entry[1])
         i = i + 1
       except:
         pass
   return()

Notice how we allow for both comma separated and space delimited data. The expectation is that the file will have two values per line, the first one being "x" and the second one being "y". They may have white space between them, or be separated by a comma. Files written this way are very common, and easy to use too, but we may not know before reading one which style it was written in. Also common (in Grace, for example), a "#" at the beginning of a line indicates a comment and implies to ignore the entire line. The reader simply skips lines that begin with "#". A more advanced reader would validate the numbers as they come in to prevent errors later. This one simply assigns them to two global arrays, one for x and one for y, because that is the format required for plotting 2D data by both matplotlib and bokeh. Also, having the data in numpy offers the options of other processing based on the GUI.

The file that is being read has been selected with a Tk widget that returns filenames in a global list

 def select_file():
   global selected_files
   
   # Use the tk file dialog to identify file(s)
   
   newfile = ""
   try:
     newfile, = tk.filedialog.askopenfilenames()
     selected_files.append(newfile)
   except:
     tk_info.set("No file selected")
   if newfile !="":
     tk_info.set("Latest file: "+newfile)
   return()

By holding onto all the selections in s list, we retain the option of going back to them later. However here in the file selection call back, we take only the first file that the user selects to add to that list. Of course we take all of them and process the one by one. The Tk function will return leaving the selected_files list with its new entry as the last one on the list, and display its name on the user interface.


Matplotlib from Tk on the Desktop


For matplotlib we need

import matplotlib as mpl import matplotlib.pyplot as plt mpl.use('TkAgg')

The Plot button call back uses matplotlib with its pyplot namespace to create a plot on the matplotlib canvas. The plot is not embedded in the Tk user interface in order to invoke the matplotlib toolbar, which in version 2.2 is deprecated for Tk. This solution avoids that issue, but also means that it is not possible to update the content of the displayed data through the Tk interface.

 # Create the desired plot
 def make_plot(event=None):
   
   global selected_files
   global x_axis_label
   global y_axis_label
   
   nfiles = len(selected_files)
   this_file = selected_files[nfiles-1]
   
   read_file(this_file)
   # Create the plot using bokeh
   
   this_file_basename = os.path.basename(this_file)
   base, ext = os.path.splitext(this_file_basename)
   bokeh_file = base+".html"
   
   output_file(bokeh_file)
   p = figure(tools="hover,crosshair,pan,wheel_zoom,box_zoom,box_select,reset") 
   p.line(x_data, y_data, line_width=2)
   show(p)
 # Create the desired plot with matplotlib
 def make_plot(event=None):
   
   global selected_files
   global x_axis_label
   global y_axis_label
   
   nfiles = len(selected_files)
   this_file = selected_files[nfiles-1]
   
   read_file(this_file)
   # Create the plot.
   plt.figure(nfiles)
   plt.plot(x_data, y_data, lw=3)
   plt.title(this_file)
   plt.xlabel(x_axis_label)
   plt.ylabel(y_axis_label)
   plt.show()


Input is handled through global variables, and the axis labels may be assigned through the Tk interface, though in tk_plot.py that is left for the next version.


Humidity tk.png



Bokeh from Tk in the Browser and on the Web

We include the bokeh modules needed for a basic plot

 from bokeh.plotting import figure, output_file, show

For bokeh the call back is very similar

 # Create the desired plot with bokeh
 def make_plot(event=None):
   
   global selected_files
   global x_axis_label
   global y_axis_label
   
   nfiles = len(selected_files)
   this_file = selected_files[nfiles-1]
   
   read_file(this_file)
   # Create the plot using bokeh
   
   this_file_basename = os.path.basename(this_file)
   base, ext = os.path.splitext(this_file_basename)
   bokeh_file = base+".html"
   
   output_file(bokeh_file)
   p = figure(tools="hover,crosshair,pan,wheel_zoom,box_zoom,box_select,reset") 
   p.line(x_data, y_data, line_width=2)
   show(p)


The tools are explicitly requested, unlike matplotlib which provides a tool bar that is fully populated.

Humidity bokeh.png



Running a Bokeh Server for Live Plotting of Python Data

Lastly, we arrive at the destination: a solution to interactive plotting where data are created and modified in Python, and presented to the user on the fly, with a customized and responsive interface. The interface components can be entirely in the browser, and thereby potentially offered to a web client, or they can be shared between the browser and a Python GUI, for desktop applications. There are three components:

  • Python backend, perhaps with Tk or other GUI or responding to CGi requests from another server
  • Bokeh server responding to the Python and presenting information to the browser
  • Browser client, listening to the Bokeh server directly or through a proxy, and providing data to the Python backend if needed

For details on how to set this up and its possible uses, see

Running a Bokeh Server

Several examples are offered here

demo.bokehplots.com

The first live example shown there is from "sliders.py", a copy of which is in our examples directory.


 # Load the modules
 import numpy as np
 from bokeh.io import curdoc
 from bokeh.layouts import row, widgetbox
 from bokeh.models import ColumnDataSource
 from bokeh.models.widgets import Slider, TextInput
 from bokeh.plotting import figure
 # Set up data using numpy
 N = 200
 x = np.linspace(0, 4*np.pi, N)
 y = np.sin(x)
 # Set up the display data source
 source = ColumnDataSource(data=dict(x=x, y=y))
 # Set up plot
 plot = figure(plot_height=400, plot_width=400, title="my sine wave",
 		tools="crosshair,pan,reset,save,wheel_zoom",
 		x_range=[0, 4*np.pi], y_range=[-2.5, 2.5])
 plot.line('x', 'y', source=source, line_width=3, line_alpha=0.6)
 # Set up widgets similar to Tk
 text = TextInput(title="title", value='my sine wave')
 offset = Slider(title="offset", value=0.0, start=-5.0, end=5.0, step=0.1)
 amplitude = Slider(title="amplitude", value=1.0, start=-5.0, end=5.0, step=0.1)
 phase = Slider(title="phase", value=0.0, start=0.0, end=2*np.pi)
 # Add interactive tools
 freq = Slider(title="frequency", value=1.0, start=0.1, end=5.1, step=0.1)
 # Set up callbacks to the widgets
 def update_title(attrname, old, new):
     plot.title.text = text.value
 text.on_change('value', update_title)
 def update_data(attrname, old, new):
     # Get the current slider values
     a = amplitude.value
     b = offset.value
     w = phase.value
     k = freq.value
     # Generate the new curve
     x = np.linspace(0, 4*np.pi, N)
     y = a*np.sin(k*x + w) + b
     source.data = dict(x=x, y=y)
 for w in [offset, amplitude, phase, freq]:
     w.on_change('value', update_data)


 # Set up layouts and add to document
 inputs = widgetbox(text, offset, amplitude, phase, freq)
 curdoc().add_root(row(inputs, plot, width=800))
 curdoc().title = "Sliders"


Download the file sliders.py or copy the source shown, and on a computer that has Python 3 and Bokeh installed, use the command line

 bokeh server sliders.py

to initiate a live session in the server. Once that has started, open your browser to "localhost", that is to your own computer, by entering this on the browser source line

 http://localhost:5006/sliders

The display will look like this, except the sliders will cause changes in the plot.


Sliders.png



Running a Server for Javascript in a Browser Engine

Python includes packages that enable a simple webserver which may be used to run advanced graphics operations through javascript within a browser's javascript engine. We will cover use of javascript, and Three.js in particular, as a supplement or replacement for 3D visualization in Python. In order to do this without the burden of managing a full Apache installation, we turn to Python. This shell script in Linux will start a web server in the directory that the script is running in:

 python -m CGIHTTPServer 8000 1>/dev/null 2>/dev/null &
 echo "Use localhost:8000"
 echo

By using port 8000 the server is distinct from the one on port 80 used for web applications. The site would appear by putting

 http://localhost:8000

in a Google Chrome or Mozilla Firefox browser window running on the same user account on the same machine. Note the redirects for stdio and stderr to /dev/null keeps output from appearing in the console. The server may be killed by identifying its process ID in Linux with the command

 ps -e | grep python

followed by

 kill -s 9  pid

where "pid" is the ID number found in the first line. Alternatively, if it is the only python process running you may kill it with

 killall python

Any file in the directory tree below the starting directory is now accessible in the browser, and html files will be parsed to run the included javascript. If here is a cgi-bin directory at the top level, the server will see it and use it. One use of this low level server is to create a virtual instrument that is accessible from the web, but not exposed to it directly. A remote web server on the same network that can access port 8000 on the instrument machine can run code and get response from the instrument by calling cgi-bin operations.

For programmers, however, this utility allows development and debugging of web software without the need for a large server.