cudf.read_json#

cudf.read_json(path_or_buf, engine='auto', orient=None, dtype=None, lines=False, compression='infer', byte_range=None, keep_quotes=False, storage_options=None, *args, **kwargs)#

Load a JSON dataset into a DataFrame

Parameters
path_or_buflist, str, path object, or file-like object

Either JSON data in a str, path to a file (a str, pathlib.Path, or py._path.local.LocalPath), URL (including http, ftp, and S3 locations), or any object with a read() method (such as builtin open() file handler function or StringIO). Multiple inputs may be provided as a list. If a list is specified each list entry may be of a different input type as long as each input is of a valid type and all input JSON schema(s) match.

engine{{ ‘auto’, ‘cudf’, ‘cudf_legacy’, ‘pandas’ }}, default ‘auto’

Parser engine to use. If ‘auto’ is passed, the engine will be automatically selected based on the other parameters. See notes below.

orientstring

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

Indication of expected JSON string format. Compatible JSON strings can be produced by to_json() with a corresponding orient value. The set of possible orients is:

  • 'split' : dict like {index -> [index], columns -> [columns], data -> [values]}

  • 'records' : list like [{column -> value}, ... , {column -> value}]

  • 'index' : dict like {index -> {column -> value}}

  • 'columns' : dict like {column -> {index -> value}}

  • 'values' : just the values array

The allowed and default values depend on the value of the typ parameter.

  • when typ == 'series',

    • allowed orients are {'split','records','index'}

    • default is 'index'

    • The Series index must be unique for orient 'index'.

  • when typ == 'frame',

    • allowed orients are {'split','records','index', 'columns','values', 'table'}

    • default is 'columns'

    • The DataFrame index must be unique for orients 'index' and 'columns'.

    • The DataFrame columns must be unique for orients 'index', 'columns', and 'records'.

typtype of object to recover (series or frame), default ‘frame’

With cudf engine, only frame output is supported.

dtypeboolean or dict, default None

If True, infer dtypes for all columns; if False, then don’t infer dtypes at all, if a dict, provide a mapping from column names to their respective dtype (any missing columns will have their dtype inferred). Applies only to the data. For all orient values except 'table', default is True.

convert_axesboolean, default True

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

Try to convert the axes to the proper dtypes.

convert_datesboolean, default True

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

List of columns to parse for dates; If True, then try to parse datelike columns default is True; a column label is datelike if

  • it ends with '_at',

  • it ends with '_time',

  • it begins with 'timestamp',

  • it is 'modified', or

  • it is 'date'

keep_default_datesboolean, default True

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

If parsing dates, parse the default datelike columns.

numpyboolean, default False

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

Direct decoding to numpy arrays. Supports numeric data only, but non-numeric column and index labels are supported. Note also that the JSON ordering MUST be the same for each term if numpy=True.

precise_floatboolean, default False

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

Set to enable usage of higher precision (strtod) function when decoding string to double values (pandas engine only). Default (False) is to use fast but less precise builtin functionality

date_unitstring, default None

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

The timestamp unit to detect if converting dates. The default behavior is to try and detect the correct precision, but if this is not desired then pass one of ‘s’, ‘ms’, ‘us’ or ‘ns’ to force parsing only seconds, milliseconds, microseconds or nanoseconds.

encodingstr, default is ‘utf-8’

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

The encoding to use to decode py3 bytes. With cudf engine, only utf-8 is supported.

linesboolean, default False

Read the file as a json object per line.

chunksizeinteger, default None

Not GPU-accelerated

This parameter is only supported with engine='pandas'.

Return JsonReader object for iteration. See the line-delimited json docs for more information on chunksize. This can only be passed if lines=True. If this is None, the file will be read into memory all at once.

compression{‘infer’, ‘gzip’, ‘bz2’, ‘zip’, ‘xz’, None}, default ‘infer’

For on-the-fly decompression of on-disk data. If ‘infer’, then use gzip, bz2, zip or xz if path_or_buf is a string ending in ‘.gz’, ‘.bz2’, ‘.zip’, or ‘xz’, respectively, and no decompression otherwise. If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression.

byte_rangelist or tuple, default None

GPU-accelerated

This parameter is only supported with engine='cudf'.

Byte range within the input file to be read. The first number is the offset in bytes, the second number is the range size in bytes. Set the size to zero to read all data after the offset location. Reads the row that starts before or at the end of the range, even if it ends after the end of the range.

keep_quotesbool, default False

GPU-accelerated feature

This parameter is only supported with engine='cudf'.

This parameter is only supported in cudf engine. If True, any string values are read literally (and wrapped in an additional set of quotes). If False string values are parsed into Python strings.

storage_optionsdict, optional, default None

Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib.request.Request as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec.open. Please see fsspec and urllib for more details.

Returns
resultSeries or DataFrame, depending on the value of typ.

See also

cudf.DataFrame.to_json

Notes

When engine=’auto’, and line=False, the pandas json reader will be used. To override the selection, please use engine=’cudf’.

Examples

>>> import cudf
>>> df = cudf.DataFrame({'a': ["hello", "rapids"], 'b': ["hello", "worlds"]})
>>> df
        a       b
0   hello   hello
1  rapids  worlds
>>> json_str = df.to_json(orient='records', lines=True)
>>> json_str
'{"a":"hello","b":"hello"}\n{"a":"rapids","b":"worlds"}\n'
>>> cudf.read_json(json_str,  engine="cudf", lines=True)
        a       b
0   hello   hello
1  rapids  worlds

To read the strings with additional set of quotes:

>>> cudf.read_json(json_str,  engine="cudf", lines=True,
...                keep_quotes=True)
          a         b
0   "hello"   "hello"
1  "rapids"  "worlds"

Reading a JSON string containing ordered lists and name/value pairs:

>>> json_str = '[{"list": [0,1,2], "struct": {"k":"v1"}}, {"list": [3,4,5], "struct": {"k":"v2"}}]'
>>> cudf.read_json(json_str, engine='cudf')
        list       struct
0  [0, 1, 2]  {'k': 'v1'}
1  [3, 4, 5]  {'k': 'v2'}

Reading JSON Lines data containing ordered lists and name/value pairs:

>>> json_str = '{"a": [{"k1": "v1"}]}\n{"a": [{"k1":"v2"}]}'
>>> cudf.read_json(json_str, engine='cudf', lines=True)
                a
0  [{'k1': 'v1'}]
1  [{'k1': 'v2'}]

Using the dtype argument to specify type casting:

>>> json_str = '{"k1": 1, "k2":[1.5]}'
>>> cudf.read_json(json_str, engine='cudf', lines=True, dtype={'k1':float, 'k2':cudf.ListDtype(int)})
    k1   k2
0  1.0  [1]