NutShell Code

Java version

• JavaDoc

Python version

Main module – nutshell.nutshell

This module contains ProductServer class, which receives product requests and forwards them to product generators. A ProductServer instance also manages disk resources defined in Configuration.

The module uses nutshell.product for defining products (nutshell.product.Info) and nutshell.request for generating them using nutshell.request.Generator .

HTTP server provided by nutshell.httpd essentially forwards HTTP requests to nutshell.ProductServer.

class nutshell.nutshell.ProductServer(conffile='')

Service designed for generating image and data products served as files

ensure_output_dir(outdir)

Creates a writable directory, if non-existent

Currently, uses mask 777

classmethod get_arg_parser(parser=None)

Populates parser with options of this class

get_cache_root()

Return the abolute path (Path) to CACHE_ROOT directory.

get_input_list(product_info, directives, log)

Used for reading dynamic input configuration generated by input.sh. directives determine how the product is generated.

get_product_dir(product_info)

Return the directory containing product generator script (generate.sh) and possible configurations etc

get_storage_root()

Return the abolute path (Path) to CACHE_ROOT directory.

init_path(dirname, verify=False)

Expand relative path to absolute, optionally check that exists.

make_prod(pr, directives=None, TEST=False)

Main function.

Parameters:pr[nutshell.product.Info] – description of the product

make_request(product_info, instructions=['MAKE'], directives=None, log=None)

Main function.

Parameters:

• product_info – description of the product (string or nutshell.product.Info)
• instructions – what should be done about the product (MAKE, DELETE, CHECK , RETURN), see Commands and status codes .
• directives – how the product is generated etc
• log – optional logging.logger

Returns:

Instance of product.Generator that contains the path of the file (if succefully generated) and information about the process.

query_file(pr)

Check if file exits or is under generation.

A file is interpreted as being under generation if a corresponding, “relatively new” empty file exists.

Parameters:pr[nutshell.product.Info] – description of the product (string or nutshell.product.Info)

read_conf(conffile=None)

Read given conf file, if it exists. Raise error, if strict. The entries are copied to respective member of self.

HTTP Server – nutshell.httpd

Server can be started with command:

python3  -m nutshell.httpd -c nutshell.cnf

By default, it returns a status page (HTML) containing

• query form
• error messages
• product info
• input info
• server status

class nutshell.httpd.NutHandler(*args, directory=None, **kwargs)

Forwards a HTTP request to nutshell.ProductServer instance

do_GET()

Main function. Receives the http request and sends a response.

static parse_url(url, data=None)

Splits url to path (str) and query data (dict)

An empty data dict is returned if query is empty.

nutshell.httpd.run_http(product_server)

A convenience function for starting and stopping the HTTP server.

Product definitions and requests – nutshell.product

class nutshell.product.Info(product=None, filename=None, product_id=None, **kwargs)

Stores product information, especially PRODUCT_ID, TIMESTAMP (if applicable), file FORMAT, and product-specific free parameters.

Parameters handled as with Info.set_product(). Key parameters are mutually exclusive.

Examples of allowed calls, with Info = nutshell.product.Info :

>>> p = Info(filename="my.image.product.png")
>>> p = Info(product_id="my.image.product")
>>> p = Info("my.image.product", **{"SIZE": "640,400"})

A plain string argument is ambiguous, and raises an exception:

>>> p = Info("my.image.product")       # Looks understandable, but...
>>> p = Info("my.image.product.jpeg")  # ... is this a product ID or file?
>>> p = Info("my.image.product_.jpeg") # This could be parsed as a file.

This class is essentially a string parser and storage for parsed variables. It is technically independent of server configurations like system side paths. Especially, it does not check if product generation directories or scripts exist.

classmethod get_arg_parser(parser=None)

Populates parser with options of this class

set_format(extension)

Sets file format (png, txt, pgm.gz, txt.zip, …).

set_id(product_id)

Set the product ID string consisting of alphanumeric chars and periods.

set_parameter(key, value='')

Set any parameter, including FORMAT, excluding TIMESTAMP and ID

Parameters:

• key – name (string) of the parameter
• value – the value of the parameter, possibly not string but hopefully stringifiable

set_parameters(params)

Given a dictionary of parameters, set values.

set_product(product=None, filename=None, product_id=None, **kwargs)

Configure a product.

Parameters:

• product – product id string, requires explicit kwargs
• product_id – product id string.
• filename – product description presented as a filename

Returns:

instance (possibly incomplete, to be adjusted with separate method calls)

See :ref:nutshell.product.Info. Parameters equivalent at initialisation.

set_timestamp(timestamp)

Set UTC time in numeric format ‘%Y%m%d%H%M’, ‘LATEST’, or ‘TIMESTAMP’

The 12 digit-numeric format may contain punctuation as long as the order of the time units is not changed. Non-digits will be simply removed. For example, 2020/03/29 18:45 is pruned to 202003291845. Consequently, possible time zones will be also discarded.

Future versions may support for time object, unix seconds and date string parsing.

Product requests – nutshell.request

class nutshell.request.Generator(product_server, product_info, log=None)

Container for storing information on requested product and server side resources derived thereof.

get_input_list(directives)

Used for reading dynamic input configuration generated by input.sh. directives determine how the product is generated.

path = PosixPath('/tmp')

System-side full path to a dynamic directory and the generated product file.

path_static = PosixPath('/tmp')

Optional: System-side full path to the generated product file.

path_tmp = PosixPath('/tmp')

Futue extension: resulting object (for example, python Image)

product_info = None

Specification of a product instance.

product_server = None

Server assigned for manufacturing this product

remove_files()

Remove the existing (even empty) files, log as info().

set_status(status)

Set success or failure status using http.HTTPStatus codes. Setting is logged.

class nutshell.request.InputQuery(product_server, product_info)

class nutshell.request.Tasklet(product_server, product_info, script_filename, log=None)

Intermediate class for shell operations (InputQuery and Generator)

Escape from Python – nutshell.shell

This module contains utilities for running shell scripts.

class nutshell.shell.Task(script, env=None, log=None)

Something that has a script, stdin, stdout, env, and log.

run(logfile_basename=None, logfile_level=40, directives=None)

Runs a task object containing task.script and task.stdout

Parameters:

• logfile_basename – Leading part of logs, ‘.err’ and ‘.out’ will be added.
• logfile_level – Save stderr and stdout to logs, if this threshold is at least logging.ERROR and logging.DEBUG, respecively
• directives[dict] – Additional instructions to the script

Returns:

Return code of the process. Will be also stored to self.

Utilities – nutshell.nutils

Utilities for handling objects and configuration files.

nutshell.nutils.debug_dict(d)

dictionary to a string.

nutshell.nutils.dict_str(d, format=' {0}="{1}"\n')

Convert a dictionary to a string.

nutshell.nutils.get_entries(obj, regexp=None, dst_type=None)

Gets member values as a dictionary

nutshell.nutils.make_subdirs(rootdir, subdir=None, mode=493)

Creates a writable directory, if non-existent :param rootdir[pathlike]: :param subdir[str]: currently, uses mask 777

nutshell.nutils.print_dict(d, format=' {0}="{1}"\n')

Dump a dictionary to stdout.

nutshell.nutils.read_conf(path, result=None)

Read plain-text configuration file consisting of <key>=<value> pairs.

nutshell.nutils.read_conf_text(text, result=None)

Traverse array of text lines consisting of <key>=<value> pairs.

nutshell.nutils.set_entries(obj, entries=None, lenient=True)

Given a dictionary, sets corresponding member values

HTML utilities – nutshell.nutxml

Utilities for main modules of NutShell

class nutshell.nutxml.KeyValueRowGenerator

Tool for generating elements that display <key>=<value> pairs for example as table rows or list items.

classmethod get(entry)

Entry should be an array or tuple.

nutshell.nutxml.get_by_id(html, id, tag='span')

tag: if create