A quickstart anaconda environment cheatsheet for everyday use.

Image for post
Image for post
Photo by Lucas Benjamin on Unsplash

Creating an Environment

We choose to create an environment named myenv

Within The Conda Folder

conda create --name myenv

Custom Location

A custom location (e.g. this directory under the name myenv)

conda create --prefix ./myenv python=3.7 --yes;

Cloning an Existing Env

conda create --name myenv --clone env2clone

Activating / Deactivating an Environment

Listing all environments

conda info --envs

To activate an environment

conda activate myenv

To activate a custom located environment

Activating an environment at location ./myenv

conda activate ./myenv

To deactivate an environment

conda deactivate

Exporting/Importing Environments

Saving your existing environment dependencies

conda env export > environment.yml

Creating a new environment from a YAML file

conda env create -f environment.yml

Conclusion

And there you have it, most of the everyday conda env commands you will ever need. If you have any additional ones, feel free to add them in the comments.


With the introduction of two-factor authentication, we are now required to change the way we access our repositories. There are now two main methods to upload to Github: using ssh keys or access tokens. This page describes the latter.

Image for post
Image for post
Photo by Markus Spiske on Unsplash

Creating a token

  1. Click on your user icon (top-right)
  2. Select Settings
  3. From the left-hand menu, select Developer Settings
  4. Personal Access Tokens
  5. Right-top: Generate New Token

Copy and SAVE the token, you will not get to view it again without generating a new one!

Storing the token

Caching for a TEMPORARY period in time

If we wish to store our token for a fixed period in time we may cache it with a predefined timeout.


How to create a fully customised colour mapping within matplotlib.

Image for post
Image for post
Photo by David Pisnoy on Unsplash

Problem

Our data is split into non-uniform categories, where a linear colour bar will be inefficient. In this tutorial, we cover how to create a custom colourmap with colours matching bins of our chosen sizes.

Solution: Discrete Colour Bars

We start by importing the required packages

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np

Next, we decide on our colours:

colours = ['#ffbe0b', '#fb5607', '#ff006e', '#8338ec', '#3a86ff']

And the bin sizes which correspond to them. — e.g. {0 → .3}, {0.3 → .7 } etc… I have intentionally chosen non-uniform bins.


Just some notes on getting started.

Image for post
Image for post
Photo by Carson Masterson on Unsplash

What is Bede?

Bede is the N8’s Tier 2 Power and GPU-based high-performance computing (HPC) platform.

Resources available

32x Main GPU nodes, each node (IBM AC922)

  • 512GB DDR4 RAM
  • 2x IBM POWER9 CPUs (and two NUMA nodes), with
  • 4x NVIDIA V100 GPUs (2 per CPU)
  • Each CPU is connected to its two GPUs via high-bandwidth, low-latency NVLink interconnects (helps if you need to move lots of data to/from GPU memory)

4x Inferencing Nodes:

  • Equipped with T4 GPUs for inference tasks.

Other

  • 2x Visualisation nodes
  • 2x Login nodes

Filestores

The filestore layout is similar to that of the ARC system. …


Notes on how to set up an interactive GPU session for computation on the Leeds System.

Image for post
Image for post
Photo by Pascal Brändle on Unsplash

Start a session

Arc4

qrsh -l h_rt=2:0:0,coproc_v100=1,h_vmem=8G -pty y bash

A single V100 card, 10 CPU cores and 48GB system memory (one quarter of the available resource on a V100 node)

Arc3

qrsh -l h_rt=2:0:0,coproc_k80=1 -pty y bash

A single K80 card, 12 CPU cores and 64GB system memory (half the available resource on a K80 node)

or

qrsh -l h_rt=2:0:0,coproc_p_100=1 -pty y bash

A single P100 card, 6 CPU cores and 64GB system memory (one quarter of the available resource on a P100 node)

Check the Card you Have Been Allocated

module load cudanvidia-smi…

Sometimes we require multiple modules all of which communicate using the I2C channel. In this post, we discuss how to do exactly that using available GPIO pins.

Image for post
Image for post
Photo by Josh Spires on Unsplash

Setup

We begin with installing the required libraries,

sudo apt-get install -y python-smbus i2c-tools

followed by enabling I2C using the advanced tab within raspi config. I this does not make sense, have google ‘using I2C on Raspberry Pi’ and the device you are trying to connect. Make sure you can connect this successfully before continuing.

Conventional I2C ports

Generally, to set up we use pins 3 and 4 (see the diagram below) to connect a device. In…


You have a suite you wish to run, but need to constrain it such that it can run on your allocated queues — this guide explains how to break it into segments which allow you to do so.

Image for post
Image for post
Photo by Djim Loic on Unsplash

Accessing the run initialisation panel through rose

The run I am using is u-ca218 . First, you need to open the rose editor and navigate to suite conf → run initialisation and cycling using the menu on the left.

Here you will have a set of options including an initialisation file, calendar and other information.


Various two-factor authentication sites now require the use of an RSA key. For me, it is needed to log into the MONSooN supercomputer. Although this is often on my desk, it is still difficult to read when flat. For this reason, I created a stand and will detail the process here.

Image for post
Image for post

Start by measuring your current key

We begin by measuring our key. This is done in mm and the use of calipers can produce much more accurate results than a simple rule.

The main dimensions of my RSA key are approximately 20mm x 10mm with the screen being 5mm from the top and bottom.

Since…


Say you have branches for different aspects of the same project and you want to merge only a single file (e.g. the configuration file). Short of copy-pasting all the changes by hand, how might you approach this problem?

Image for post
Image for post
Photo by The Nigmatic on Unsplash

The Problem

So we have a branch. This branch contains an update, bug-fix or enchantment and we wish to pass that on to all our other branches. This can be for reasons that you want to keep a testing copy or share information between code that no longer has the same function between different parts of the same repository.

Here merging the entire body…


A guide to trigger a data transfer or other process when inserting a USB in Linux

computer with USB ports — cover image
computer with USB ports — cover image
Photo by Tobias Lystad on Unsplash.

In our latest Raspberry Pi sensor project, we wanted to have a no-network approach to getting data automatically off it. The solution for this came through the running of a Python script on insertion of an “authorised” USB storage device.

In this article, I will explain how such a feat may be achieved.

The USB Rules

The simplest part lies in providing a set of rules on what to do upon insertion. These consist of a script for when we insert a USB and one for when we remove it:

ACTION=="add", SUBSYSTEM=="usb", PROGRAM="<full_path_here>/on_usb_in.sh"
ACTION=="remove", SUBSYSTEM=="usb", PROGRAM="<full_path_here>/on_usb_out.sh"

These two lines go into the etc/udev/rules.d

Daniel Ellis

Research Software Engineer specialising in Data Visualisation with a touch of HPC. — PhD in Atmospheric Chemistry and Masters in Theoretical Physics.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store