Scripts of the pyveg package

pyveg.scripts.analyse_gee_data module

This script analyses data previously download with download_gee_data.py. First, data is preprocessed using the analysis_preprocessing.py module. Plots are produced from the processed data.

pyveg.scripts.analyse_gee_data.analyse_gee_data(input_location, input_location_type='local', input_json_file=None, output_dir=None, do_time_series=True, do_spatial=False, upload_to_zenodo=False, upload_to_zenodo_test=False)

Run analysis on dowloaded gee data

Parameters

• input_location (str) – Location of results_summary.json output from pyveg_run_pipeline, OR if input_location_type is zenodo or zenodo_test, the 2-digit coordinate_id representing the row in coordinates.py.

• input_location_type (str) – Can be ‘local’, ‘azure’, ‘zenodo’, or ‘zenodo_test’.

• input_json (str, optional. Full path to the results summary json file.) –

• output_dir (str,) – Location for outputs of the analysis. If None, use input_location

• do_time_series (bool) – Option to run time-series analysis and do plots

• do_spatial (bool) – Option to run spatial analysis and do plots

• upload_to_zenodo (bool) – Upload results to the production Zenodo repo

• upload_to_zenodo_test (bool) – Upload results to the sandbox Zenodo repo

pyveg.scripts.analyse_gee_data.main()

CLI interface for gee data analysis.

pyveg.scripts.analyse_gee_data.run_early_warnings_resilience_analysis(filename, output_dir)

Run early warning resilience analysis on time series data. This function can be called on the detrended process data.

Parameters

• filename (str) – Path to the time series csv to analyse.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.analyse_gee_data.run_time_series_analysis(filename, output_dir, detrended=False)

Make plots for the time series data. This function can be called for the seasonal or detrended process data.

Parameters

• filename (str) – Path to the time series csv to analyse.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.analyse_pyveg_summary_data module

This script analyses summary statistics produced previously with analyse_gee_data.py for individual locations.

pyveg.scripts.analyse_pyveg_summary_data.analyse_pyveg_summary_data(input_location, output_dir)

Run analysis on summary statistics data

Parameters

input_location (str) –

Location of summary statistics output files from analyse_gee_data.py.

Can be ‘zenodo’ or ‘zenodo_test’ for the production or sandbox Zenodo depositions, or the path to the local directory containing the files.

output_dir: str,

Location for outputs of the analysis. If None, use input_dir

pyveg.scripts.analyse_pyveg_summary_data.barplot_plots(df, output_dir)

Create barplots of summary data.

Parameters

• df (dataframe) – Dataframe of summary data.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.analyse_pyveg_summary_data.boxplot_plots(df, output_dir)

Create boxplots of summary data.

Parameters

• df (dataframe) – Dataframe of summary data.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.analyse_pyveg_summary_data.correlation_plots(df, output_dir)

Create correlation plots of summary data.

Parameters

• df (dataframe) – Dataframe of summary data.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.analyse_pyveg_summary_data.main()

CLI interface for gee data analysis.

pyveg.scripts.analyse_pyveg_summary_data.process_input_data(input_location)

Read all input summary statistics and transform data into

a more analysis-friendly format

Parameters

input_location (str) – Location of summary statistics output files from analyse_gee_data.py. Can be ‘zenodo’ or ‘zenodo_test’ to download from the production or sandbox Zenodo depository, or the path to a directory on the local filesystem.

Returns

df

Return type

DataFrame containing all the time-series data concatenated together.

pyveg.scripts.analyse_pyveg_summary_data.scatter_plots(df, output_dir)

Create scatter plots and correlation plots of summary data.

Parameters

• df (dataframe) – Dataframe of summary data.

• output_dir (str) – Path to the directory to save plots to.

pyveg.scripts.calc_euler_characteristic module

script to run the Euler Characteristic calculation on a given input image. The code that does the actual calculations is in src/subgraph_centrality.py

pyveg.scripts.calc_euler_characteristic.main()

pyveg.scripts.create_analysis_report module

pyveg.scripts.create_analysis_report.add_rgb_images(mdFile, rgb_location, rgb_location_type, fig_count)

pyveg.scripts.create_analysis_report.add_time_series_plots(mdFile, analysis_plots_location, analysis_plots_location_type, satellite_suffix)

pyveg.scripts.create_analysis_report.create_markdown_pdf_report(analysis_plots_location, analysis_plots_location_type, rgb_location, rgb_location_type, do_timeseries, output_dir=None, collection_name=None, metadata=None)

pyveg.scripts.create_analysis_report.get_collection_and_suffix(collection_name)

Lookup collection and suffix based on the name of the collection as used by GEE

Parameters

collection_name (str, GEE name of the collection, eg. 'COPERNICUS/S2') –

Returns

• collection (str, user-friendly name of the collection, e.g. ‘Sentinel2’)

• suffix (str, contraction of collection name, used in the filenames of plots)

pyveg.scripts.create_analysis_report.main()

CLI interface for gee data analysis.

pyveg.scripts.crop_and_convert_images module

Use some of the functionality in src/image_utils.py as command-line script.

pyveg.scripts.crop_and_convert_images.main()

use command line arguments to specify input and output directories, and parameters for doing the cropping and conversion.

pyveg.scripts.download_from_azure module

pyveg.scripts.generate_config_file module

Generate a config file pyveg/configs/<config_filename> for use when running download and processing jobs with pyveg_run_pipeline –config_file pyveg/configs/<config_filename>

User specifies: * Coordinates OR id of location in coordinates.py * Date range * time per point * Satellite collection name (e.g. “Sentinel2”, “Landsat8”) * run mode (“local” or “batch”) * whether to run in ‘test’ mode (fewer dates, and only a few sub-images).

These can be given directly as command-line arguments, or the user will be prompted for them.

Usage

pyveg_generate_config

then respond to prompts, or

pyveg_generate_config –help

to see a list of command line options. (Note that command line options and prompted inputs can be mixed-and-matched).

pyveg.scripts.generate_config_file.get_template_text()

pyveg.scripts.generate_config_file.main()

pyveg.scripts.generate_config_file.make_filename(configs_dir, test_mode, longitude, latitude, country, pattern_type, start_date, end_date, time_per_point, region_size, collection_name, run_mode, coords_id)

Construct a filename from the specified parameters.

pyveg.scripts.generate_config_file.make_output_location(coords_id, collection_name, latitude, longitude, country)

pyveg.scripts.generate_config_file.write_file(configs_dir, output_location, longitude, latitude, country, pattern_type, start_date, end_date, time_per_point, region_size, collection_name, run_mode, n_threads, test_mode=False, coords_id=None)

Take the arguments, construct a filename, and write contents

pyveg.scripts.generate_pattern module

Translation of Matlab code to model patterned vegetation in semi-arid landscapes.

pyveg.scripts.generate_pattern.main()

pyveg.scripts.plot_feature_vectors module

pyveg.scripts.run_pyveg_module module

Build and run a pyveg pipeline based on a configuration json file.

pyveg.scripts.run_pyveg_module.build_module(config_file)

Load json config and instantiate modules

pyveg.scripts.run_pyveg_module.configure_and_run_module(module)

Call configure() run() on all the module

pyveg.scripts.run_pyveg_module.main()

pyveg.scripts.run_pyveg_pipeline module

Build and run a pyveg pipeline based on a configuration json file.

pyveg.scripts.run_pyveg_pipeline.build_pipeline(config_file, from_cache=False)

Load json config and instantiate modules

pyveg.scripts.run_pyveg_pipeline.configure_and_run_pipeline(pipeline)

Call configure() run() on all sequences in the pipeline.

pyveg.scripts.run_pyveg_pipeline.main()

pyveg.scripts.upload_to_zenodo module

Upload the results_summary.json or the outputs of the time-series analysis to the Zenodo open source platform for data www.zenodo.org.

Will create a zipfile, with a name based upon the coordinates and satellite collection, and upload it to a “deposition” via the zenodo API.

In addition to the main production zenodo API, there is also a “sandbox” for testing. Use the ‘–test_api’ argument to use this.

pyveg.scripts.upload_to_zenodo.create_new_deposition(use_sandbox=False)

Create a new deposition, and populate it with the metadata from zenodo_config.py.

Parameters

use_sandbox (bool, if True use the "sandbox" API rather than the production one) –

Returns

deposition_id – is then copied into zenodo_config.py

Return type

int, the ID of the newly created deposition. Recommended that this

pyveg.scripts.upload_to_zenodo.main()

pyveg.scripts.upload_to_zenodo.upload_results_summary(json_location, json_location_type, use_test_api)

Upload the results summary json from running pyveg pipeline to download and process data from GEE.

pyveg.scripts.upload_to_zenodo.upload_summary_stats(csv_filepath, use_test_api)

Typically called by the analyse_gee_data script, upload the results summary csv file.

Module contents