Funcs

Generate randomized series of bytes which can be used as an input I/O handle.

Configure an I/O handle to use the specified charset. The handle is any supported I/O handle and the charset is a string name supported by the JVM installation. Standard charset names:

• "UTF-8" 8-bit Unicode Transformation Format
• "UTF-16BE": 16 bit Big Endian Unicode Transformation Format
• "UTF-16LE" 16 bit Little Endian Unicode Transformation Format
• "ISO-8859-1": Latin-1 code block
• "US-ASCII": 7-bit ASCII

Examples:

// write text file in UTF-16BE
ioWriteStr(str, ioCharset(`io/foo.txt`, "UTF-16BE"))

// read CSV file in ISO-8859-1
ioCharset(`io/foo.csv`, "ISO-8859-1").ioReadCsv

Convert a handle to append mode. Writes will append to the end of the file instead of rewriting the file. Raises UnsupportedErr if the handle doesn't support append mode.

Example:

ioWriteStr("append a line\n", ioAppend(`io/foo.txt`))

Read a directory listing, return a grid with cols:

• uri: Uri for handle to read/write the file
• name: filename string
• mimeType: file mime type or null if unknown
• dir: marker if file is a sub-directory or null
• size: size of file in bytes or null
• mod: modified timestamp or null if unknown

If the I/O handle does not map to a file in the virtual file system then throw an exception.

ioDir(`io/`)             // read files in project's io/ directory
ioDir(`fan://haystack`)  // read files in pod

Get information about a file handle and return a Dict with the same tags as ioDir().

If the I/O handle does not map to a file in the virtual file system then throw an exception.

ioInfo(`io/`)            // read file info for the project's io/ directory
ioInfo(`io/sites.trio`)  // read file info for the io/sites.trio file

Create a directory or empty file with the given I/O handle. If creating a file that already exists, it is overwritten as empty.

Examples:

ioCreate(`io/new-dir/`)        // create new empty directory
ioCreate(`io/new-file.txt`)    // create new empty file

Delete a file or a directory as mapped by the given I/O handle. If a directory is specified, then it is recursively deleted. If the I/O handle does map to a file system then raise exception. If the file does not exist then no action is taken.

Copy a file or directory to the new specified location. If this file represents a directory, then it recursively copies the entire directory tree. Both handles must reference a local file or directory on the file system.

If during the copy, an existing file of the same name is found, then the "overwrite" option should be to marker or true to overwrite or false to skip. Or if overwrite is not defined then an IOErr is raised.

Examples:

ioCopy(`io/dir/`, `io/dir-copy/`)
ioCopy(`io/file.txt`, `io/file-copy.txt`)
ioCopy(`io/file.txt`, `io/file-copy.txt`, {overwrite})
ioCopy(`io/file.txt`, `io/file-copy.txt`, {overwrite:false})

Move or rename a file or directory. Both handles must reference a local file or directory on the file system. If the target file already exists then raise an IOErr.

Read an I/O handle into memory as a string. Newlines are always normalized into "\n" characters.

Write a string to an I/O handle.

Read an I/O handle into memory as a list of string lines. Lines are processed according to InStream.readLine semanatics.

By default the maximum line size read is 4kb of Unicode characters (not bytes). This limit may be overridden using the option key "limit".

Examples:

ioReadLines(`io/file.txt`)
ioReadLines(`io/file.txt`, {limit: 10_000})

For each line of the given source stream call the given function with two parameters: Str line and zero based Number line number. Lines are processed according to InStream.eachLine.

Read a stream of lines. Lines are processed according to InStream.eachLine. See Streams.

Write a list of string lines separated with "\n" character.

Read a Trio file into memory as a list of Dicts.

Write dicts to a Trio file. The val may be can be any format accepted by toRecList().

Following options are supported

• noSort: marker to prevent tags from being sorted by name

Read a Zinc file into memory as a Haystack data type.

Write a Grid to the Zinc format.

Read a Xeto data file into memory as a Haystack data type. See XetoIO.readXeto for details and options.

Write value to a Xeto text format file. See XetoIO.writeXeto for details and options.

Read a CSV (comma separated values) file into memory as a Grid. CSV format is implemented as specified by RFC 4180:

• rows are delimited by a newline
• cells are separated by delimiter char
• cells containing the delimiter, " double quote, or newline are quoted; quotes are escaped as '""'
• empty cells are normalized into null

The following options are supported:

• delimiter: separator char as string, default is ","
• noHeader: if present then don't treat first row as col names, instead use "v0", "v1", etc

Also see ioStreamCsv(), ioEachCsv(), ioWriteCsv(), and CSV.

Read a stream of dicts from a comma separated value file. This function uses the same options and semantics as ioReadCsv() except it streams the rows as dicts instead of reading to an in-memory grid. See Streams.

Iterate the rows of a CSV file (comma separated values) and callback the given function with two parameters: Str[] cells of current row and zero based Number line number.

The following options are supported:

• delimiter: separator char as string, default is ","

Also ioReadCsv(), ioWriteCsv(), and CSV.

Write a grid to a CSV (comma separated values) file.

CSV format is implemented as specified by RFC 4180:

• rows are delimited by a newline
• cells are separated by delimiter char
• cells containing the delimiter, " double quote, or newline are quoted; quotes are escaped as '""'

The following options are supported:

• delimiter: separator char as string, default is ","
• newline: newline string, default is "\n" (use "\r\n" for CRLF)
• noHeader: Set this option to prevent the column names from being written as a header row.
• stripUnits: write all numbers without a unit

Also ioReadCsv(), ioEachCsv(), and CSV.

Write an Excel XLS file, where val may be:

• Grid - writted a a single worksheet
• Grid[] - each grid is exported as a separate worksheet

By default each worksheet is named "Sheet1", "Sheet2", etc. Use a title tag in Grid.meta to give the worksheets a specific name.

Example:

readAll(site).ioWriteExcel(`io/sites.xls`)

Read a JSON file into memory. Any JSON file decodes to Axon dicts, lists, strings, numbers, and bools - so this can be used to read any arbitrary JSON. Where a value carries a Hayson _kind tag it decodes to that Haystack type, so {"_kind":"date", "val":"2024-11-26"} becomes a Date while a plain string stays a string.

Prefer ioReadHayson() if decoding Hayson and not generic JSON. Also see ioReadJsonGrid() if reading a Haystack formatted grid. Use ioReadJeto() for xeto-typed JSON.

Object keys which are not valid tag names will decode correctly and can be used in-process. But they will not serialize correctly to grids/zinc. Use the safeNames option to force object keys to be safe tag names (but you will lose the original key names).

The following options are supported:

• safeNames: convert object keys to safe tag names
• v3: decode the JSON as Haystack 3 (deprecated, will be removed in the future)
• v4: explicitly request the default Haystack 4 decoding

Read a JSON file formatted as a standardized Hayson grid into memory. See ioReadJson() to read arbitrary JSON structured data.

Write an Axon data structure to JSON using the Hayson encoding, where each typed scalar is an object naming its own kind. If not serializing generic objects and strings, then prefer ioWriteHayson(). Use ioWriteJeto() for xeto-typed JSON.

The following options are supported:

• noEscapeUnicode: do not escape characters over 0x7F
• v3: Encode JSON using Haystack 3 encoding (deprecated, will be removed in the future)
• v4: Explicitly request the default Haystack 4 encoding

Read a Hayson file into memory, the Haystack 4 JSON encoding where each typed scalar carries a _kind discriminator. A value with no _kind decodes as a plain dict, list, string, number or bool.

This is the same as ioReadJson() but restricted to Hayson. It does not accept the legacy Haystack 3 option. See ioReadJeto() for xeto-typed JSON.

The following options are supported:

• safeNames: convert object keys to safe tag names

Write an Axon data structure to Hayson, the Haystack 4 JSON encoding where each typed scalar carries a _kind discriminator.

This is the same as ioWriteJson() but restricted to Hayson. It does not accept the legacy Haystack 3 option. See ioWriteJeto() for xeto-typed JSON.

The following options are supported:

• noEscapeUnicode: do not escape characters over 0x7F

Read a Jeto file into memory. Jeto is xeto-typed JSON: the spec supplies the types, so the values don't have to declare them. A string decodes as the type its spec gives it, and as a Str where no spec covers it.

The optional spec names the type expected at the root, which types the values beneath it. Without it only what the payload declares for itself is recovered. It may be a spec literal or its string qname:

ioReadJeto(`io/site.json`, {spec:Site})
ioReadJeto(`io/site.json`, {spec:"ph::Site"})

The following options are supported:

• spec: the spec expected at the root as a Spec or string qname

Write an Axon data structure to Jeto. Jeto is xeto-typed JSON: the spec supplies the types, so the values don't have to declare them. A scalar which no spec covers is boxed as an object naming its own spec.

The following options are supported:

• box: "none", "auto", or "all" boxing mode
• pretty: add indentation to pretty print

Write an Axon data structure to HTML. The val must be an Axon type that can be converted to a Grid.

Write a grid to an XML file.

Write xeto instance data to RDF Turtle format. The val must be a dict, list of dicts, or grid of instances and every instance requires an 'id' tag.

The following options are supported:

• baseUri: mint IRIs for refs which are not library-qualified as '<{baseUri}{ref}>'; defaults to the runtime's API rec base under the http extension's site URI

Examples:

// write sites using the default rec base, so a rec id such as
// @p:demo:r:abc mints as <https://host/api/demo/rec/p:demo:r:abc>
readAll(site).ioWriteRdf(`io/sites.ttl`)

// mint the same rec id as <https://acme.com/rec/p:demo:r:abc>
readAll(site).ioWriteRdf(`io/sites.ttl`, {baseUri:`https://acme.com/rec/`})

Read a zip file's entry listing, return a grid with cols:

• path: path of entry inside zip as Uri
• size: size of file in bytes or null
• mod: modified timestamp or null if unknown

The handle must reference a zip file in the file system. Use ioZipEntry() to perform a read operation on one of the entries in the zip file.

Example:

ioZipDir(`io/batch.zip`)

Return a I/O handle which may be used to read from a zip entry within a zip file. The handle parameter must be an I/O handle which references a zip file in the file system. The path parameter must be a Uri which identifies the path of the entry within the zip file. See ioZipDir() to read the listing of paths within a zip.

Example:

// read CSV file from within a zip
ioZipEntry(`io/batch.zip`, `/zone-temp.csv`).ioReadCsv

Wrap an I/O handle to GZIP compress/uncompress.

Example:

// generate GZIP CSV file
readAll(site).ioWriteCsv(ioGzip(`io/sites.gz`))

// read GZIP CSV file
ioGzip(`io/sites.gz`).ioReadCsv

Return an I/O handle to decode from a base64 string. Also see ioToBase64() and Buf.fromBase64

Example:

// decode base64 to a string
ioFromBase64("c2t5c3Bhcms").ioReadStr

Encode an I/O handle into a base64 string. The default behavior is to encode using RFC 2045 (see Buf.toBase64). Use the {uri} option to encode a URI-safe URI via RFC 4648 (see Buf.toBase64Uri). Also see ioFromBase64().

Example:

// encode string to base64
ioToBase64("myusername:mysecret")

// encode string to base64 without padding using URI safe chars
ioToBase64("myusername:mysecret", {uri})

Encode an I/O handle into hexidecimal string.

Generate a cycle reduancy check code as a Number. See Buf.crc for available algorithms.

Example:

ioCrc("foo", "CRC-32").toHex

Generate a one-way hash of the given I/O handle. See Buf.toDigest for available algorithms.

Example:

ioDigest("foo", "SHA-1").ioToBase64

Generate an HMAC message authentication as specified by RFC 2104. See Buf.hmac.

Example:

ioHmac("foo", "SHA-1", "secret").ioToBase64

Generate a password based cryptographic key. See Buf.pbk.

Example:

ioPbk("PBKDF2WithHmacSHA1", "secret", ioRandom(64), 1000, 20).ioToBase64

Apply a skipping operation to an input I/O handle. The following options are available (in order of processing):

• bom: skip byte order mark
• bytes: number of bytes to skip (must be binary input stream)
• chars: number of chars to skip (must be text input stream)
• lines: number of lines to skip

Skipping a BOM will automatically set the appropiate charset. If no BOM is detected, then this call is safely ignored by pushing those bytes back into the input stream. The following byte order marks are supported:

• UTF-16 Big Endian: 0xFE_FF
• UTF-16 Little Endian: 0xFF_FE
• UTF-8: 0xEF_BB_BF

Examples:

// skip leading 4 lines in a CSV file
ioSkip(`io/foo.csv`, {lines:4}).ioReadCsv

// skip byte order mark
ioSkip(`io/foo.csv`, {bom}).ioReadCsv

Render data to a PDF file. The grid meta "view" tag determines the visualization:

• table: render grid as table (default)
• chart: render grid as chart
• fandoc: render string as fandoc
• text: render as plaintext

Options:

• pageSize: determines the PDF page sizePrinted page size formatted as "width,height" with an explicit unit of "pt", "in", "mm", or "cm". Examples "8.5in,11in" or "612pt,792pt"

Examples:

// render as chart with default page size
read(power).hisRead(yesterday).ioWritePdf(`io/portrait.pdf`)

// render as chart with landscape page size of 11" x 8.5"
read(power).hisRead(yesterday).ioWritePdf(`io/landscape.pdf`, {pageSize:"11in,8.5in"})

// render table as single auto-fit page
readAll(site).ioWritePdf(`io/sites.pdf`, {pageSize:"auto"})

Note: this feature is available in SkySpark only

Render data to an SVG file. Pass size option to specify the SVG viewBox, width, and height attribtues (defaults to "1000,800"). The visualization is determined by the grid meta "view" tag - see ioWritePdf() for specifics.

Examples:

read(power).hisRead(yesterday).ioWriteSvg(`io/example.svg`)

read(power).hisRead(yesterday).ioWriteSvg(`io/example.svg`, {size:"600,400"})

Note: this feature is available in SkySpark only

Export a view to the given file handle - see hx.doc.fresco::Export.

Note: this feature is available in SkySpark only

Make an HTTP request and process the response in a callback. The connection is guaranteed to be closed when the callback returns.

The headers parameter is an optional Dict of request headers. Header values may use the special ref @password to resolve a secret from the project's password store. The password is looked up by the folllowing case sensitive key (raises exception if not found):

<scheme>://<host>[:<port>]/ <header-name>

The body parameter is any value accepted as an I/O handle (Str, Buf, Uri, etc). If non-null, it is streamed as the request body; Content-Type defaults to "application/octet-stream" when not specified in headers.

The callback receives three arguments:

• code: Number - HTTP status code
• headers: Dict - response headers with names normalized to lower-case. If the response set one or more cookies via Set-Cookie, they are collapsed and reformatted as a "set-cookie" entry using valid Cookie request header syntax ("name=value; name2=value2") so it can be passed directly back into a subsequent ioHttp call's headers under the "Cookie" key without any parsing. Header names are not valid tag names, so use toTagNames() to return or inspect the headers as a grid.
• body: response body I/O handle for streaming reads

External API responses often use JSON keys that aren't valid Haystack tag names (e.g. containing spaces, dashes, or leading digits) — use the {safeNames} option on ioReadJson() to convert them automatically rather than risk a decode error.

Examples:

// GET and read response as string
ioHttp(`http://example.com/api`, "GET", null, null,
  (code, headers, body) => ioReadStr(body))

// POST with body
ioHttp(`http://example.com/api`, "POST",
  {"Content-Type": "text/plain"}, "hello world",
  (code, headers, body) => {code: code, body: ioReadStr(body)})

// Optionally use the password store for auth header
passwordSet("https://acme.com/ Secret-Header", "pass")

// Reference that password in the headers
ioHttp(`https://acme.com/api`, "POST",
  {"Secret-Header": @password, "Content-Type": "application/json; charset=utf-8"},
  body,
  (code, headers, body) => {code: code, body: ioReadJson(body, {safeNames})})

// Carry a session cookie from one request to the next
r1: ioHttp(`https://acme.com/login`, "POST", null, credsBody,
  (code, headers, body) => {code: code, cookie: headers["set-cookie"]})

ioHttp(`https://acme.com/data`, "GET", {"Cookie": r1->cookie}, null,
  (code, headers, body) => ioReadJson(body, {safeNames}))