Developer Guidelines


These notes are designed to detail best practice in using the various APIs available in Dawn as a programmer. With Dawn one can either add new features using python scripts with pydev, using workflows or with Java in the form of osgi plugins. The python API and workflows are user level since any user may extend Dawn this way without changing Dawn. The Java programmer will normally need to contribute to the github repos and will need to rebuild Dawn before the feature is available in Dawn or will be using Dawn APIs in their own project, such as GDAfor instance. This guide is designed for Java programmers and explains some of the APIs available and when to use them.

If you are familiar with eclipse and buckminster, this quick start guide should get you up and running with developing DAWN. In order to push your changes, please complete a ‘Pull Request’ on github. There are four simple points to follow to get up and running:

1. Eclipse and Buckminster

If not already installed you will need the JDK installed. You will need to install a version of eclipse for ‘RCP and RAP developers’ and also add the Buckminster update site, something like: depending on version. (It is not required to install git as you can develop using egit inside eclipse acceptably well.)
Alternatively, you can Install an eclipse bundle put together at Diamond that comes with all the necessary plugins (eGit, Buckminster etc) which can be downloaded here.

2. Clean workspace

Create a folder to be the workspace. Inside this folder expand the most recent zip file from here:

This ensures that the defaults and target are going to be the same for all developers of DAWN.

3. CQuery

Open eclipse with the folder you created as the workspace folder. We are now going to run a Buckminster query to pull over all the DAWN source code. From the file menu in eclipse choose 

‘Open a component query’ and enter the URI Then enter the ‘eclipse.feature’ of On the properties of the CQuery, change “download.location” to “public” and if you don’t have ssh keys for github setup, set “github.authentication” to “anonymous”

Then press the ‘Resolve and Materialize’ button.

Figure 1

Figure 1 CQuery used to materialize a DAWN workspace

DAWN development environment will now materialize to your folder and a folder at the same level appended with ‘_git’ for the git repos.

4. Compile and Run

Please turn on ‘Build Automatically’ from the project menu. DAWN will now compile, don’t worry if there seem to be errors at this stage. When we run the product these will resolve.

Go to the plugin ‘org.dawnsci.product.plugin’ and open the org.dawnsci.base.product. Start DAWN from the toolbar at the top which contains a ‘Launch an Eclipse application in Debug mode’ action.

Figure 2

Figure 2 Compile and run DAWN

Congratulations, you are now able to develop and debug DAWN. Happy hacking…


The plotting is a feature rich and elegant API which as well as plotting data can manage tools and selection regions. 

@Deprecated Do not use DataSetPlotter for plotting. This class has now been deprecated and it will not be supported after Raptor.

Getting started with IPlottingSystem

The plotting system in Dawn is encapsulated behind an interface (feature rich and well documented, see javadocs). This interface is called IPlottingSystem. If you are doing any plotting in Dawn, you should use this interface. For Dawn 1.0 and 1.1 the interface includes 1D and 2D plotting, multiple axes, multiple regions (replaces overlays in the old system), configuration of almost everything, real time / interactive plots, advanced histogramming etc. From Dawn Raptor onwards, 3D will be included. 

Unlike the plotting system in SDA, the default implementation of IPlottingSystem (known as ‘Light Weight Plotting System’) will pass over NX clients and other remote client technologies. In addition it manages the toolbar actions automatically (these can be manipulated in createPartControl of your part as well). Also it is no longer necessary to manage the low level drawing outside the plotting package for regions. A feature rich set of regions are provided in the API.

A Minimum View to Plot Something in DAWN
public class AViewToPlotSomething extends ViewPart {
        private IPlottingSystem plotting;
	private Logger logger = LoggerFactory.getLogger(AViewToPlotSomething.class);
	public AViewToPlotSomething() {
		try {
			this.plotting = PlottingFactory.createPlottingSystem();
		} catch (Exception e) {
			logger.error("Cannot create a plotting system!", e);

	public void createPartControl(Composite parent) {
		plotting.createPlotPart(parent, "My Plot Name", getViewSite().getActionBars(), PlotType.IMAGE, this);

	public void setFocus() {

	public Object getAdapter(final Class clazz) {
		if (IPlottingSystem.class == clazz)
			return plotting;
		if (IToolPageSystem.class == clazz)
			return plotting;
		return super.getAdapter(clazz);

Other examples can be found in the project org.dawnsci.plotting.examples in repo dawn-ui.

Plotting Something

When you would like to plot some data there are thread safe convenience methods the full ITrace API. The ITrace API allows more control but must be completed in the UI thread. There are numerous calls to the thread safe and ITrace based methods throughout Dawn which can be used as example code. Also some simple cases are detailed below. (Other examples can be found in the project org.dawnsci.plotting.examples in repo dawn-ui.)

Thread Safe Convenience Methods
@see IPlottingSystem.createPlot1D(...)   // Add new plot
@see IPlottingSystem.updatePlot1D(...)  // Update trace with same name if there or create new if not
@see IPlottingSystem.createPlot2D(...)   // Add new plot
@see IPlottingSystem.updatePlot2D(...)  // Update trace with same name if there or create new if not
//(From Raptor onwards)
@see IPlottingSystem.createPlot3D(...)  // Add new plot
@see IPlottingSystem.updatePlot3D(...)  // Update trace with same name if there or create new if not


IPlottingSystem system = ...
// Later in a job or thread:

// Make some random data
final LongDataset ls = createRandomDataset(2048);
AbstractDataset x = AbstractDataset.arange(ls.getSize(),AbstractDataset.INT32);

// Plot something
system.createPlot1D(indices, Arrays.asList(ls), monitor); // monitor may be null.
ITrace API

The low level or ‘Trace’ API allows all plots, in 1D, 2D or 3D to be dealt with using the ITrace object. Each ITrace object can be created, modified, added and removed. This must all be done from the UI thread. The order operation is generally:

Line Traces for 1D

The plots for 1D can be dealt with by the ILineTrace class. An example of using this follows:

final String traceName = "fred";
final ILineTrace trace = getPlottingSystem().createLineTrace(traceName);
trace.setUserObject(...); // Some data I kept to know that it was my plot, optional

trace.setData(getXdata(), getYdata()); // AbstractDatasets x and y
trace.setTraceColor(new Color(null, getPlotColour()));

system.repaint(); // optional, can be done at end of adding many.
Image Traces for 2D

Plotting images can be done in a similar way to plotting lines, using the standard ‘create, configure, add’ methodology. An example is:

final IImageTrace image = system.createImageTrace("fred");
image.setMask(...); // AbstractDataset of same size as data.
image.setData(AbstractDataset, List<AbstractDataset> axes, false);

Making Image Traces Update Fast

This rate is around ~20 ms / 2k image i.e. ~50 Hz / 2k image on an intel i7. The image is downsampled so a larger image would be measuring the speed of the downsampling algorithm.

To get this speed you need these lines on IImageTrace:

    imageTrace.setDownsampleType(DownsampleType._POINT_); // Fast!
    imageTrace.setRescaleHistogram(**false**); // Fast!

Then to send the data you need, this must be done in the UI thread.

    imageTrace.setData(data, null, false);
Surface Traces for 2D/3D

From Raptor onwards Dawn will begin to support more 3D operations in the plotting API. An example of this is the new ISurfaceTrace which is accessible using system.createSurfaceTrace(…).


The system is event based - meaning that when almost anything changes in the plotting it can be listened to. For instance new traces plotted by the user, new regions added, regions dragging etc. Some useful listeners and their function are explained below:

 Listener  Function

 ITraceListener  Notification of when traces are about to be plotted, have been added, have been deleted etc. The TraceWillPlot event allows the data of the trace to be modified which can be a useful advanced feature.

 IRegionListener  Notification of IRegions being created, add and deleted.

 IROIListener  Added to IRegions this listener notifies of location. Do not do work in the drag notification which takes a lot of CPU or your drag will be very slow!

 IPaletteListener  Can be added to an IImageTrace to be notified of a range of image properties being changed.

 IToolChangeListener  Notification of tools being changed by the user.

 IAxisListener  Changes to the ranges of Axes. Added to an IAxis.

 ICoordinateSystemListener  Added to an ICoordinateSystem to be notified of coordinate changes.


There are a number of useful services which are available in Dawn. These can be retrieved in eclipse 3.x using:

IDawnService service = (IDawnService)PlaformUI.getWorkbench().getService(IDawnService.class);

The following table explains their usage (@see plugin for all the other services available):

 Service  Function

 IImageService Service for getting Images (ImageData) for any 2D AbstractDataset.

 ILoaderService Service for loading data (using LoaderFactory)

 IPaletteService Service for getting registered palettes (there is an extension point for registering an Image palette).

 IThumbnailService Service for getting a thumbnail image of an dataset. 

 ISystemService Service for getting plotting systems backed by PlottingFactory.

For example:

final ISystemService<IPlottingSystem> service = (...)PlatformUI.getWorkbench().getService(ISystemService.class);  

 IPersistenceService Service for getting an IPersistentFile used to save and load various data. This service allows the saving of data, regions, masks to the HDF5 file format. (OSGI service only, PlaformUI.getWorkbench().getService(…) cannot be used.)

 IConversionService  Service for converting/exporting files of one format to another. For instance export a 3D hdf5/nexus stack to a directory of TIFF images.(OSGI service only, PlaformUI.getWorkbench().getService(…) cannot be used.)

Loader Service / Loader Factory

The ILoaderService is backed by LoaderFactory and either the ILoaderService or the LoaderFactory (which has static methods and more options for loading) are acceptable to use for loading data into Dawn. Please do not use loader classes (e.g. SRSLoader) directly either in Jython or Java code. It is much better to use the factory because it can determine the right loader automatically and it has a soft data cache for caching recently used data. This approach has a significant speed advantage.

Adding a FileLoader

Adding a file loader to be picked up by ILoaderService (and LoaderFactory) can be done using an extension point in Dawn 1.x. 

To do this create a plugin for your loader and use the extension point ”” to provide a class and a file extension for loading a given format. You can also set the priority. Since multiple file loaders for a given file extension are allowed, the priority provides a way of specifying when the loader should be tried, usually set to high for a custom file loader. Your loader should fail throwing an exception for invalid files, as soon as possible. This will make the LoaderService perform in a faster more robust way.

For example:

    <extension point="">
	<loader class="org.dawb.gda.extensions.loaders.H5Loader" file_extension="h5, nxs, hdf5, hd5, hdf, nexus" high_priority="true">

Your new file loader class must extend AbstractFileLoader ( and optionally (usually) implement IMetaLoader and IDataSetLoader. If the file format can load information about the data faster than parsing all the data, it is a very good idea to implement IMetaLoader. @see org.dawb.gda.extensions.loaders.H5Loader

Thumbnail Service

The thumbnail service should be used when pulling out large numbers of thumbnails from a directory of images. It has been optimized in speed and memory for doing this task.

Persistence Service

The persistence service can be used to save and load data, regions, masks to and from HDF5 files. The persistence service can be obtained like the following:

    IPersistenceService service = (IPersistenceService)ServiceManager.getService(IPersistence.class);  

Then in order to save data the method service.createPersistenceFile(filePath) needs to be called.It returns an IPersistentFile that will be used save data using setters. To read data, service.getPersistenceFile(filePath) has to be used with the getters available to the IPersistentFile.

Once the reading/saving is done, the IPersistentFile needs to be closed:
Reading ROIs Example:

IPersistentFile file = service.createPersistenceFile(filePath);  
... = file.getROIs();  


One of the more important concepts in Dawn are regions. These replace the old overlays in SDA and integrate elegantly with the region of interest (or ROI) code from the mathematics plugin of Dawn ( The regions follow the same methodology for creation programmatically:

    region = createRegion(...);
    // The plotting system mouse will change to show that clicking in the plot will start to add the region, or you can continue and add the region programmatically

Common region types:

Region Type Default color ROI preferred
LINE(“Line”, ColorConstants.cyan, LinearROI.class),
POLYLINE(“Polyline”, ColorConstants.cyan, PolylineROI.class),
POLYGON(“Polygon”, ColorConstants.cyan, PolygonalROI.class),
BOX(“Box”,, RectangularROI.class),
GRID(“Grid”, ColorConstants.lightGray, GridROI.class),
CIRCLE(“Circle”, darkYellow, CircularROI.class),
SECTOR(“Sector”,, SectorROI.class),
POINT(“Point”, darkMagenta, PointROI.class),
ELLIPSE(“Ellipse”, ColorConstants.lightGreen, EllipticalROI.class),
ELLIPSEFIT(“Ellipse fit”, ColorConstants.lightGreen, EllipticalFitROI.class),
RING(“Ring”, darkYellow, SectorROI.class),
XAXIS(“X-Axis”,, RectangularROI.class),
YAXIS(“Y-Axis”,, RectangularROI.class),
XAXIS_LINE(“X-Axis Line”,, RectangularROI.class),
YAXIS_LINE(“Y-Axis Line”,, RectangularROI.class),
FREE_DRAW(“Free draw”, darkYellow, PolylineROI.class);


Tool Pages

Tools can be added to Dawn using the extension point “org.dawb.common.ui.toolPage”. You should declare a class extending AbstractToolPage and some auxiliary information such as if it is a 1D or 2D tool and the icon to use.

There is an example tool in the package, the version of ExampleTool at the time of writing is shown below.


import org.dawb.common.ui.plot.tool.AbstractToolPage;
import org.dawb.common.ui.util.GridUtils;
import org.eclipse.swt.SWT;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Control;
import org.eclipse.swt.widgets.Display;

public class ExampleTool extends AbstractToolPage {

	private Composite control;
	public ExampleTool() {
		// Create your listeners to the main plotting 
		// Perhaps create a plotting system here from the PlottingFactory which is your side plot.

	public ToolPageRole getToolPageRole() {
		return ToolPageRole.ROLE_1D;

	public void createControl(Composite parent) {
		this.control = new Composite(parent, SWT.NONE);
		control.setLayout(new GridLayout(1, false));
		// User interface shown in a page to the side of the plot.
		// ... For instance: a side plot, a Viewer part

	public Control getControl() {
		return control;

	public void setFocus() {
		// If you have a table or tree in your tool, set focus here.

	public void activate() {
		// Now add any listeners to the plotting providing getPlottingSystem()!=null

	public void deactivate() {
		// Now remove any listeners to the plotting providing getPlottingSystem()!=null

	public void dispose() {
		// Anything to kill off? This page is part of a view which is now disposed and will not be used again.

Tool Actions

It is possible to contribute actions to tools either directly when programming the tool but also through eclipse extension points. The extension point for contributing actions to a tool is called “org.dawb.common.ui.toolPageAction”. The action requires the id of the tool and the id of the command that the action should run. Other options such as Icon and where to place the tool are possible.  NOTE this system is separate to the eclipse system for adding actions by extension point.

Tools and Perspectives

Tools are noramlly chosen by the user via the plotting menu to open a tool. Users may also choose to keep a tool open while opening other tools by opening the tool in a dedicated view. Another option is for the creator of a perpective to open the tool by adding it directly to a perspective. This is done in the IPerspectiveFactory implementation by opening a view with an id of “org.dawb.workbench.plotting.views.toolPageView.fixed” and a secondary id equal to the tool concerned.

For instance to open the diffraction tool fixed in a perspective (as is done in the diffraction perspective we have:

IFolderLayout toolPageLayout = layout.createFolder(....);

Another example to open the region editor in a view would be:



Any workbench part containing plotting may also slice data from files with multi-dimensional datasets (like nexus/df5 for instance). Here is the recipe for doing this:

  1. Your view or editor with plotting must implement ISlicablePlottingPart. It has one method:
    void updatePlot(final ITransferableDataObject[] selections, final ISliceSystem sliceSystem, final boolean useTask);

Where selections is a list of data which has already been sliced and is ready to plot. 

Implement the method to use getDataset(…) on ITransferableDataObject to obtain a reference of the IDataset required for plotting. 

  1. You must implement the following one or two clauses in your IWorkbenchPart.getAdpater(…)
    public Object getAdapter(@SuppressWarnings("rawtypes") final Class clazz) {
	if (clazz == Page.class) {
		return PlotDataPage._getPageFor_(this);
	// If your part is not an IEditorPart
	if (clazz == IFile.class) {
		return ...;// the IFile which contains the data
	return super.getAdapter(clazz);
  1. Include the ‘Data’ view in your perspective, id = ”org.dawb.workbench.views.dataSetView”


You can now slice any data that your part can access using the standard DAWN slicing.

Figure 3

Figure 3 Slicing UI in DAWN

If this UI is too complex we can create simpler modes for the slice component as required.

HDF5 Libraries

Simple HDF5 Reading

The standard reading uses ILoaderService or LoaderFactory if you are on the GDA Server (and OSGI is not available).

final ILoaderService service = (ILoaderService)PlatformUI.getWorkbench().getService(ILoaderService.class);
IDataHolder holder = service.getData(filePath, new ProgressMonitorWrapper( monitor));
ILazyDataset lz = holder.getLazyDataset("/entry/path/to/my/data");

The data can now be sliced to get parts of the data. Also lz.getSlice() will read all of the data (use with caution).

Using the Factory (Expert use only)

There is a HierarchicalDataFactory for getting read ability of an HDF5 File from any thread. This attempts to deal with any threading issues with reading the file in a multi-threaded environment. DAWN and GDA user interfaces are multi-threaded and this is an issue because background threads may run to decorate or pull out other information from HDF5 files while the workbench is running.

HDF5 has a number of complexities in getting working with Java programs. Firstly there are raw HDF5 APIs (two, one high level and one low level) and there is a Nexus API including some JNI for the nexus format. 

The Nexus JNI should be avoided. It is adds an additional C++ JNI layer which can result in the system crashing, it is better to do more work in the Java layer using the HDF5 apis. In addition it forces some types of read to be blocking when the HDF5 API is non-blocking. In further addition, its version lags behind HDF5 which has regular releases with often quite powerful features (such as the forthcoming multi-threaded write).

HDF5 can read in a multi-threaded way however only one file handle must exist per h5 file being read (threads can share one file handle for a file but cannot have many handles for the same file). The solution to this is not to use synchronization on the read but to use caching of paths and synchronized access to the file handle. Synchronization on the read results in a slower to read system with is not acceptable for data analysis.

The recommended HDF5 API is IHierarchicalDataFile. This supports multi-threaded read and single threaded write. It supports writing of nexus attributes. It uses the high level HDF5 API from HDFView so is well tested. It exposes the low level HDF5 API for more advanced things such as chunking.


Example 1 pull out data set names of a given type:

final IHierarchicalDataFile hFile = HierarchicalDataFactory._getReader_(h5.getAbsolutePath());
try {
	final List<String> names = hFile.getDatasetNames(IHierarchicalDataFile._NUMER_ARRAY_);
	if (names.size()!=noResults)
		throw new Exception("Unexpected data sets from test file '"+afile+"' in hdf5 file '"+h5.getName()+"' they were: "+names);
	// ...
	} finally {

Example 2 use the factory to get low level HDF5 objects:

IHierarchicalDataFile file = null;
try {
	file = HierarchicalDataFactory.getReader(getFilePath());
	Group grp = (Group)file.getData(FUNCTION_ENTRY);
	if (grp==null) 
		throw new Exception("Reading Exception: " +FUNCTION_ENTRY+ " entry does not exist in the file " + filePath);
	List<HObject> children = grp.getMemberList();
	if (names==null) names = new ArrayList&lt;String&gt;(children.size());
	for (HObject hObject : children) {
} finally {
	if (file!=null) file.close();

Example 3 Write datasets to an HDF5 file:

IHierarchicalDataFile file = null;
try {
	file = HierarchicalDataFactory.getReader(getFilePath());
	final Group entry ="entry"); // Creates a group if needed, also, "child")
	file.setNexusAttribute(entry, Nexus._ENTRY_);

	final AbstractDataset a = ...;
	final Datatype d = H5Utils.getDatatype(a);

	final long[] shape = H5Utils.getLong(a.getShape());

	final Dataset s = file.createDataset(a.getName(), d, shape, a.getBuffer(), entry);
	file.setNexusAttribute(s, Nexus.SDS);
	// There is also file.appendDataset(...) for adding to a H5 stack
} finally {
	if (file!=null)

Using the Low Level API (God use only)

You can really get problems with using or copying code that uses the low level API. This is because we are talking about JNI here and if you use the calls in the wrong order or fail to close an entry in a try {} finally{} you can force the Java VM to exit. If you have to use the low level API, here is a top tip wrap you code in a try {} finally{} in the following way:

// Lock for low level in case anyone else reads:
try {
	// Open in the low level using something like:
	int fapl = H5.H5Pcreate_(HDF5Constants.H5P_FILE_ACCESS);
	//.. etc
	try {
		// Do your evil low level access
	} finally {
		// Do all your low level closing in a finally
} finally {

Connecting Plot to Python

If you want any IPlottingSystem to connect to python to be scriptable, you should use the ScriptConnection class from

the plugin To use it in a workbench part do the following:

  1. Where you create your plotting system and after you have attached most of your listeners etc. (for instance createPartControl(…) ), put the lines:
this.connection = new ScriptingConnection(getPartName()); // Or whatever name the scripting should use connection.setPlottingSystem(plottingSystem);
  1. Remember to dispose the connection:
    public void dispose() {
     if (plottingSystem!=null)

Unit/Regression Testing

Automated testing in Dawn takes three forms currently:

  1. Junit tests
  2. Junit plugin tests
  3. Squish UI tests

The junit and squish tests are run on the Jenkins build system. If a developer checks in code to Dawn or changes the build to break these tests, it is expected that they will also investigate why this has occurred and take appropriate action. A change may be reverted if it breaks the tests for a prolonged period of time.

The junit plugin tests are not currently run by Jenkins. These tests are run by hand before a release. They include memory leak tests and should all pass before release.

General Eclipse RCP stuff

How to contribute to the DAWN welcome/intro page

Coming soon

Vertical and horizontal scroll bars on DAWN views

When creating a new view, it is important to take into account how the users will end up using this view. The machine which DAWN will be running on is also an important issue. Indeed if DAWN is being used on a laptop, chances are that most perspectives won’t be properly shown given the small dimension of the screen. To make sure that the user experience is not downgraded it is recommended to implement views with vertical and horizontal scrollbars so that main UI components such as buttons, text fields or other widgets won’t disappear if the available screen space is too small.

To do so a ScrolledComposite can be implemented:

final ScrolledComposite scrollComposite = new ScrolledComposite(parent, SWT.H_SCROLL | SWT.V_SCROLL);  
final Composite content = new Composite(scrollComposite, SWT.NONE);  
content.setLayout(new GridLayout(1, false));  
content.setLayoutData(new GridData(SWT.FILL, SWT.FILL, true, true));  

// some widget with content as a parent composite  
scrollComposite.addControlListener(new ControlAdapter() {
	public void controlResized(ControlEvent e) {  
		Rectangle r = scrollComposite.getClientArea();  
		int height = content.computeSize(r.width, SWT.DEFAULT).y;  
		scrollComposite.setMinWidth(content.computeSize(SWT.DEFAULT, r.height).x);