Runtime Validation

With TypedDict, any static type checker that already supports TypedDicts can be leveraged to check the classes generated by ts2python.

However, there are use-cases where dynamic type checking of TypedDicts might be relevant. For example, when processing json-data stemming from an external source which might happen to provide invalid data.

Also, up to Python 3.10 TypedDict does not allow marking individual items as required or not required. (See PEP 655 for the details.) Static type checkers that do not evaluate the Required and NotRequired annotation will produce false results for TypedDicts that contain not-required fields.

Module ts2python.json_validation provides functions and function annotations to validate (arbitrarily nested) typed dicts. In order to use runtime type-checking, ts2python.json_validation provides shims for TypedDict, GenericTypedDict and NotRequired that should be imported instead of Python’s typing.TypedDict and typing.GenericTypedDict. Runtime json-Validation can fail with obscure error messages, if the TypedDict-classes against which values are checked at runtime do not derive from ts2python.json_validation.TypedDict!

Validation with decorators

The easiest way to use runtime type checking is by adding the json_validation.type_check()-annotation to a function receiving or returning a TypedDict:

>>> from ts2python.json_validation import TypedDict, type_check
>>> class Position(TypedDict, total=True):
...     line: int
...     character: int
>>> class Range(TypedDict, total=True):
...     start: Position
...     end: Position
>>> @type_check
... def line_too_long(rng: Range) -> bool:
...     return (rng['start']['character'] > 255
...             or rng['end']['character'] > 255)
>>> line_too_long({'start': {'line': 1, 'character': 1},
...                'end': {'line': 8, 'character': 17}})
False
>>> try:
...     line_too_long({'start': {'line': 1, 'character': 1},
...                    'end': 256})
... except TypeError as e:
...     print(e)
Parameter "rng" of function "line_too_long" failed the type-check, because:
Type error(s) in dictionary of type <class '__main__.Range'>:
Field end: '256' is not of <class '__main__.Position'>, but of type <class 'int'>

By default the json_validation.type_check()-annotation validates both the arguments of a function and its return value. (This behaviour can be configured with the check_return_type-parameter of the annotation.) Type validation will not take place on arguments or return values for which no type annotation is given.

validate_type-function

Alternatively, types can be validated by the calling json_validation.validate_type(). validate_type() does not return anything but either raises a TypeError if the given value does not have the expected type:

>>> from ts2python.json_validation import validate_type
>>> validate_type({'line': 42, 'character': 11}, Position)
>>> try:
...     validate_type({'line': 42, 'character': "bad mistake"}, Position)
... except TypeError as e:
...     print(e)
Type error(s) in dictionary of type <class '__main__.Position'>:
Field character: 'bad mistak...' is not a <class 'int'>, but a <class 'str'>

The validate_type- and the type_check-annotation will likewise complain about missing required fields and superfluous fields:

>>> from ts2python.json_validation import NotRequired
>>> class Car(TypedDict, total=True):
...     brand: str
...     speed: int
...     color: NotRequired[str]
>>> @type_check
... def print_car(car: Car):
...     print('brand: ', car['brand'])
...     print('speed: ', car['speed'])
...     if 'color' in car:
...         print('color: ', car['color'])
>>> print_car({'brand': 'Mercedes', 'speed': 200})
brand:  Mercedes
speed:  200
>>> print_car({'brand': 'BMW', 'speed': 180, 'color': 'blue'})
brand:  BMW
speed:  180
color:  blue
>>> try:
...     print_car({'speed': 200})
... except TypeError as e:
...     print(e)
Parameter "car" of function "print_car" failed the type-check, because:
Type error(s) in dictionary of type <class '__main__.Car'>:
Missing required keys: {'brand'}
>>> try:
...     print_car({'brand': 'Mercedes', 'speed': 200, 'PS': 120})
... except TypeError as e:
...     print(e)
Parameter "car" of function "print_car" failed the type-check, because:
Type error(s) in dictionary of type <class '__main__.Car'>:
Unexpected keys: {'PS'}

Type validation works its way up from the root type down to any nested object. Type unions, e.g. int|str are evaluated by trying all alternatives on the data until one alternative matches. Enums and uniform sequences (e.g. List[str]) are properly taken care of.

Reference

Module json_validation.py - provides validation functions and decorators for those types that can occur in a JSON-dataset, in particular TypedDicts.

Copyright 2021 by Eckhart Arnold (Eckhart.Arnold@badw.de)

Bavarian Academy of Sciences an Humanities (badw.de)

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

json_validation.type_check(func: Callable, check_return_type: bool = True) → Callable

Decorator that validates the type of the parameters as well as the return value of a function against its type annotations during runtime. Parameters that have no type annotation will be silently ignored by the type check. Likewise, the return type. Example:

>>> class Position(TypedDict, total=True):
...     line: int
...     character: int
>>> class Range(TypedDict, total=True):
...     start: Position
...     end: Position
>>> # just a fix for doctest stumbling over ForwardRef:
>>> Range.__annotations__ = {'start': Position, 'end': Position}
>>> @type_check
... def middle_line(rng: Range) -> Position:
...     line = (rng['start']['line'] + rng['end']['line']) // 2
...     character = 0
...     return Position(line=line, character=character)
>>> rng = {'start': {'line': 1, 'character': 1},
...        'end': {'line': 8, 'character': 17}}
>>> middle_line(rng)
{'line': 4, 'character': 0}
>>> malformed_rng = {'start': 1, 'end': 8}
>>> try:
...     middle_line(malformed_rng)
... except TypeError as e:
...     print(e)
Parameter "rng" of function "middle_line" failed the type-check, because:
Type error(s) in dictionary of type <class 'json_validation.Range'>:
Field start: '1' is not of <class 'json_validation.Position'>, but of type <class 'int'>
Field end: '8' is not of <class 'json_validation.Position'>, but of type <class 'int'>

Parameters:

func – The function, the parameters and return value of which shall be type-checked during runtime.

Returns:

The decorated function that will raise TypeErrors, if either at least one of the parameter’s or the return value does not match the annotated types.

json_validation.validate_type(val: Any, typ)

Raises a TypeError if value val is not of type typ. In particular, validate_type() can be used to validate dictionaries against TypedDict-types and, more generally, to validate JSON-data. Examples:: >>> validate_type(1, int) >>> validate_type([‘alpha’, ‘beta’, ‘gamma’], List[str]) >>> class Position(TypedDict, total=True): … line: int … character: int >>> import json >>> json_data = json.loads(‘{“line”: 1, “character”: 1}’) >>> validate_type(json_data, Position) >>> bad_json_data = json.loads(‘{“line”: 1, “character”: “A”}’) >>> try: … validate_type(bad_json_data, Position) … except TypeError as e: … print(e) Type error(s) in dictionary of type <class ‘json_validation.Position’>: Field character: ‘A’ is not a <class ‘int’>, but a <class ‘str’>

json_validation.validate_uniform_sequence(sequence: Iterable, item_type)

Ensures that every item in a given sequence is of the same particular type. Example:

>>> validate_uniform_sequence((1, 5, 3), int)
>>> try:
...     validate_uniform_sequence(['a', 'b', 3], str)
... except TypeError as e:
...     print(e)
3 is not of type <class 'str'>

Parameters:

• sequence – An iterable to be validated

• item_type – The expected type of all items the iterable sequence yields.