Run the steps of a project

Reads the project contract, plans the selected steps and publishes their tables under path.tables, file by file, returning the record of what it published. The steps are named after the table each one publishes and always run in the order ShearTable, DnTable, kmaxTable.

Usage

processProject(
  file = "newmark.json",
  root,
  steps = c("ShearTable", "DnTable", "kmaxTable"),
  sites = character(),
  dryRun = FALSE
)

Arguments

file

Path of the JSON contract, resolved against root when it is relative.

root

Existing project root against which the contract and its relative paths are resolved; required, with no default. The CLI passes the working directory when --root is absent.

steps

Any of "ShearTable", "DnTable" and "kmaxTable"; they run in that order.

sites

Identifiers of the sites to process; empty selects every site found under path.uhs. Only DnTable and kmaxTable process sites, and an unknown site stops them before anything is written. The global tables are rebuilt from the tables of every site found, selected or not.

dryRun

TRUE makes the checks that a run makes before its first calculation (see Details), reports with message() the contract, the steps, the sites and every destination it would write or replace, and returns without calculating, seeding, creating a directory or writing anything.

Value

Invisibly, the effects record: a data.table with one row per published file in publication order (with dryRun, one per destination) and the columns step, siteID (NA for a global table), product, path (the destination spelled from path.tables as the contract declares it, so relative to root when that path is relative), action ("writes" when the destination did not exist, "replaces" when it did) and rows (the rows of the published table; NA in a dry run).

Details

Everything the run decides is resolved before its first calculation, and a dry run makes the same checks: the contract keys of the selected steps, the paths, the sites and the destinations. For ShearTable, the plan also applies to the geometries, the materials, the pairs and Vref.gmdp the rules that buildShearTable() applies to its arguments, and to the USCS codes of every selected material the rule of sampleSoilLayers(). For DnTable and kmaxTable, the inputs that no selected step produces must exist, and what the shear table and the MCE table hold is checked before those steps calculate anything: for DnTable, the mean rows of the shear table and the MCE demand of every selected site over the period 1.5 * max(Ts) of those rows, as buildDnTables() requires them; for kmaxTable, the heights Hs from which buildKmaxTable() derives its displacements. A dry run makes the checks of DnTable first and a run the check of the heights first, as each phase of newmark 1.7.1 does, so that when both fail the two report different failures. With ShearTable selected as well, the shear table is the one that step publishes: a run makes these checks after publishing it, and a dry run, which publishes nothing, does not make them. A contract or an input that these checks reject stops the run before the step it belongs to writes anything, in a dry run as in a run.

Project contract

The contract is one JSON object with the keys of newmark 1.7.1 (schema 1), validated strictly. Keys are looked up exactly, and a key outside the list below stops the run naming it; that includes Hs.min, Hs.max, NTS, KMAX_PARALLEL and KMAX_WORKERS, which 1.7.1 read. Each key is required only by the steps named with it. file and the paths of the path block are resolved against root when they are relative; the working directory is never changed and reading evaluates no code. Numbers are read as doubles whatever their JSON spelling, so 760 and 760.0 are the same value.

path.tables

Directory that newmark writes, and the only one: the global tables at its root and the tables of each site under <ID.gmdp>/<siteID>/. Every step; ShearTable creates it when it is absent.

path.data

Directory of the hazard table MCETable.Rds, read only. DnTable.

path.uhs

Directory holding <ID.gmdp>/<siteID>/UHSRock.Rds, read only; the sites are its directories that hold one. DnTable and kmaxTable.

path.calc

JSON of the classical hazard calculation, whose finite positive job.calculation.truncation_level truncates the sampling of the rock demand. DnTable.

Vref.gmdp

Reference rock velocity in m/s, one number. Every step; DnTable and kmaxTable require 760.

ID.gmdp, TR.gmdp, NS, subduction, Dn_weights

Identifier of the hazard model (a letter followed by letters, digits, _ or -), return periods in years (sorted, duplicates dropped), realizations per cell (a positive whole number), the regime (true for subduction) and the 0/1 switches of the displacement models, which selectDisplacementModels() turns into the weights of the active models. DnTable and kmaxTable.

Mw.gmdp

Moment magnitude of the scenarios. DnTable.

DaH.gmdp

Admissible displacements, distinct positive percentages of the height Hs. kmaxTable.

DN_PARALLEL, DN_WORKERS

Optional parallel controls of DnTable: true (the default) or false, and a positive whole number of workers, by default the cores that R detects minus one, at least one. A string, a logical or a fraction is rejected. The workers evaluate the geometry and material pairs of a demand, so the step never uses more of them than the mean rows of the shear table, and false keeps it in one process; the tables do not depend on them.

uscs, IDm

Object of USCS code arrays indexed by material, and the distinct selected materials, each one a key of uscs. ShearTable.

Hs, lo.min, lo.max, s.min, s.max

Range form of the geometries: the heights in metres, and the first and last truncation ratio and slope, spread evenly over the heights in their declared order (they are not a minimum and a maximum) and passed to buildGeometry(). ShearTable, unless geometryTable is declared; the heights have no default.

geometryTable

Optional array of IDg, Hs, b and s row objects, used instead of the range form.

pairs

Optional array of IDg and IDm row objects restricting the product of geometries by materials.

seed

Optional whole number between 0 and 2147483647 that seeds the ShearTable step. A contract without it, such as one written for newmark 1.7.1, runs that step on the caller's random stream, as 1.7.1 does without a seed, and the contract alone does not make its table reproducible.

Steps

ShearTable builds the geometries, calls set.seed() with seed when the contract defines it, which replaces the caller's random stream, calls buildShearTable() with the levels 0.02, 0.05, 0.10, 0.16, mean, 0.84, 0.90, 0.95 and 0.98, and publishes <path.tables>/ShearTable.Rds with saveRDS(). Its message reports the seed, or seed none without one.

DnTable reads ShearTable.Rds under path.tables (the one the ShearTable step has just published, when both run), MCETable.Rds and the UHSRock.Rds of each selected site, which it processes in alphabetical order. For a site it calls buildDnTables() with ID.gmdp, TR.gmdp, Vref.gmdp, Mw.gmdp, NS, the weights that selectDisplacementModels() makes of Dn_weights and subduction, the truncation level of path.calc and the workers, and buildPGATable(), and publishes under <path.tables>/<ID.gmdp>/<siteID>/, in this order, DnDraws.Rds with saveRDS(), PGATable.csv with data.table::fwrite() and na = "NA", and DnTable.Rds with saveRDS(). It then rebuilds the global DnPlotTable.Rds and DnTable.Rds with assembleDnTables() from the DnTable.Rds of every site found under path.uhs that has one, selected or not, and publishes them, in that order, at the root of path.tables; a site found without that file stays out of them. buildDnTables() seeds each site and demand with its own fixed seed, so the step replaces the caller's random stream and the tables of a site do not depend on the sites run with it.

kmaxTable reads ShearTable.Rds under path.tables and, for each selected site in alphabetical order, its DnDraws.Rds and its PGATable.csv, read back with data.table::fread() and TR numeric. For a site it calls buildKmaxTable() with ID.gmdp, TR.gmdp, NS, the weights that selectDisplacementModels() makes of Dn_weights and subduction, and DaH.gmdp, and publishes <path.tables>/<ID.gmdp>/<siteID>/kmaxTable.Rds with saveRDS(). It then assembles the global kmaxTable.Rds with assembleKmaxTables() from the kmaxTable.Rds of every site found under path.uhs that has one, selected or not, without reading their PGATable.csv, and publishes it at the root of path.tables; a site found without kmaxTable.Rds stays out of it. The warnings of both functions about roots beyond the stored grid reach the caller, and the step draws no random number.

The inputs of the site steps that no selected step produces must exist when they are planned: ShearTable.Rds under path.tables, MCETable.Rds for DnTable, and the DnDraws.Rds and PGATable.csv of each selected site for kmaxTable without DnTable.

Publication

The tables are published in groups: the shear table; the three files of a site and the two global files of DnTable; the site and the global kmaxTable. Every file of a group is first written to a temporary file .<name>.<suffix> beside its destination, in the format its name declares, and then each destination is replaced by renaming its temporary file, in the order of the group. Each rename is atomic; the group is not, and nothing is rolled back. A failure while writing the temporary files changes no file of the group; a failure at the rename of a file leaves the files before it replaced and the others with their previous content, or still absent; a failure in a later calculation or check leaves everything published before it: the checks of the site steps that follow a published shear table, the calculation of a later site, or a table of another site that the global tables cannot take (a DnTable.Rds that assembleDnTables() rejects, or a kmaxTable.Rds that assembleKmaxTables() rejects), which stops the step with the selected sites published and the global files unchanged. The temporary files are removed when the function exits, by return or by error, but not when the R process ends between two renames. Running a step again with the same inputs publishes the same tables, except ShearTable without seed.

An error raised before the first file is published reaches the caller unchanged, as the same condition object; that includes the error of the writer of a temporary file. An error raised after it, while publishing, calculating or validating, is signalled as a condition of class newmarkPartialPublication. Its message names the step and site that failed, the destination when the failure was a publication, the original message, the files already published and the files of the failing group that keep their previous content; its field cause holds the original condition, effects the effects record of the files already published, and path the destination whose temporary file or rename failed (NULL when the failure was not a publication). A rename that fails returns FALSE, with at most a warning, so it is reported as an error naming its destination and the reason.

Examples

Root <- file.path(tempdir(), "newmarkProject")
dir.create(Root)
writeLines('{"path": {"tables": "data/newmark"}, "Vref.gmdp": 760,
  "Hs": [10, 12], "lo.min": 0.35, "lo.max": 0.075, "s.min": 2.5, "s.max": 3.5,
  "uscs": {"U1": ["ML", "MH"]}, "IDm": ["U1"], "seed": 20260915}',
  file.path(Root, "newmark.json"))
print(processProject(root = Root, steps = "ShearTable", dryRun = TRUE))
#> Dry run of /tmp/RtmpLIBqKv/newmarkProject/newmark.json
#> Steps: ShearTable
#> ShearTable writes data/newmark/ShearTable.Rds
#>          step siteID    product                        path action  rows
#>        <char> <char>     <char>                      <char> <char> <int>
#> 1: ShearTable   <NA> ShearTable data/newmark/ShearTable.Rds writes    NA
# \donttest{
Effects <- processProject(root = Root, steps = "ShearTable")
#> ShearTable wrote ShearTable (18 rows, seed 20260915)
Effects
#>          step siteID    product                        path action  rows
#>        <char> <char>     <char>                      <char> <char> <int>
#> 1: ShearTable   <NA> ShearTable data/newmark/ShearTable.Rds writes    18
readRDS(file.path(Root, Effects$path))[level == "mean"]
#>       Hs     b     s  Vref    IDm    IDg   POP    Hw   USCS    Go    mo    Ts
#>    <num> <num> <num> <num> <char> <char> <num> <num> <char> <num> <num> <num>
#> 1:    10    27   2.5   760     U1    S10   100     0  ML MH  72.8 0.427 0.179
#> 2:    12     7   3.5   760     U1    S12   100     0  ML MH  80.1 0.432 0.180
#>      VSo  VS30  Z500   Z1000  level    lo     an
#>    <num> <num> <num>   <num> <char> <num>  <num>
#> 1: 193.7 326.6  15.7 187.768   mean 0.351 3.5446
#> 2: 203.0 303.4  11.9 115.420   mean 0.077 2.8530
# }
unlink(Root, recursive = TRUE)