106. Data Validation with pydantic

Pydantic is a powerful library for data validation in Python. It allows you to define Python data models using type annotations, and automatically validates the data according to the specified types. This makes it easier to enforce data consistency and ensure that the data being processed is in the correct format.

Here are some Python code snippets demonstrating how to perform data validation using Pydantic.

1. Basic Data Validation

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

# Valid Data
person = Person(name="Alice", age=30)
print(person)

# Invalid Data (will raise validation error)
# person = Person(name="Alice", age="thirty")  # Raises error

Explanation: In this example, the Person model validates that name must be a string and age must be an integer. If you try to pass invalid data (like age="thirty"), it will raise a ValidationError.

---

2. Field Constraints with constr and conint

from pydantic import BaseModel, constr, conint

class Product(BaseModel):
    name: constr(min_length=3)
    price: conint(gt=0)

# Valid Data
product = Product(name="Laptop", price=1000)
print(product)

# Invalid Data (will raise validation error)
# product = Product(name="TV", price=-100)  # Raises error

Explanation: Here, constr(min_length=3) ensures that the product name has at least 3 characters, and conint(gt=0) ensures that the price is greater than zero.

---

3. Default Values and Optional Fields

from pydantic import BaseModel
from typing import Optional

class Book(BaseModel):
    title: str
    author: str
    published_year: Optional[int] = None

# Valid Data
book = Book(title="Python 101", author="John Doe")
print(book)

# Optional field can be omitted
book_with_year = Book(title="Python 102", author="Jane Doe", published_year=2022)
print(book_with_year)

Explanation: The Optional type allows fields to be omitted, and a default value can be provided (e.g., published_year defaults to None).

---

4. Nested Models

from pydantic import BaseModel
from typing import List

class Address(BaseModel):
    street: str
    city: str
    postal_code: str

class User(BaseModel):
    username: str
    email: str
    address: Address

# Valid Data
user = User(
    username="alice123",
    email="[e-mail]",
    address=Address(street="123 Main St", city="Wonderland", postal_code="12345")
)
print(user)

# Invalid Data (raises error)
# user = User(username="alice123", email="[e-mail]", address="invalid address")  # Raises error

Explanation: You can nest models within each other to represent more complex structures, like an address within a user model. Pydantic ensures the nested model data is also validated.

---

5. Data Validation with Regex

from pydantic import BaseModel, constr

class User(BaseModel):
    username: constr(regex=r'^[a-zA-Z0-9]{6,12}$')  # Only alphanumeric and 6-12 characters

# Valid Data
user = User(username="user123")
print(user)

# Invalid Data (raises error)
# user = User(username="user!")  # Raises error

Explanation: You can use regular expressions to enforce specific patterns for string fields, such as usernames that must be alphanumeric and between 6 to 12 characters long.

---

6. Custom Validation with @root_validator

from pydantic import BaseModel, root_validator

class Product(BaseModel):
    name: str
    price: float
    discount: float

    @root_validator
    def check_discount(cls, values):
        price = values.get('price')
        discount = values.get('discount')
        if discount >= price:
            raise ValueError("Discount cannot be greater than or equal to price")
        return values

# Valid Data
product = Product(name="Laptop", price=1000, discount=200)
print(product)

# Invalid Data (raises error)
# product = Product(name="Laptop", price=1000, discount=1200)  # Raises error

Explanation: The @root_validator decorator allows you to validate fields across the entire model. In this case, it checks that the discount cannot be greater than or equal to the price.

---

7. Enforcing Enum Types

from pydantic import BaseModel
from enum import Enum

class Role(Enum):
    admin = "admin"
    user = "user"
    guest = "guest"

class User(BaseModel):
    username: str
    role: Role

# Valid Data
user = User(username="alice123", role=Role.admin)
print(user)

# Invalid Data (raises error)
# user = User(username="alice123", role="moderator")  # Raises error

Explanation: Enums can be used to restrict a field to a predefined set of values. In this case, the role field is restricted to one of the Role enum values.

---

8. Datetime Validation

from pydantic import BaseModel
from datetime import datetime

class Event(BaseModel):
    name: str
    date: datetime

# Valid Data
event = Event(name="Python Workshop", date="2022-10-10T10:00:00")
print(event)

# Invalid Data (raises error)
# event = Event(name="Python Workshop", date="2022-13-40T10:00:00")  # Raises error

Explanation: Pydantic automatically validates datetime fields to ensure the data conforms to the correct datetime format.

---

9. Validator with Custom Logic

from pydantic import BaseModel, validator

class Product(BaseModel):
    name: str
    price: float

    @validator('price')
    def check_price(cls, value):
        if value <= 0:
            raise ValueError('Price must be greater than 0')
        return value

# Valid Data
product = Product(name="Laptop", price=1000)
print(product)

# Invalid Data (raises error)
# product = Product(name="Laptop", price=-100)  # Raises error

Explanation: The @validator decorator is used to apply custom validation logic to specific fields. In this case, it ensures the price is greater than 0.

---

10. Parsing Data with JSON Input

from pydantic import BaseModel
import json

class User(BaseModel):
    username: str
    email: str

# Valid JSON
json_data = '{"username": "alice123", "email": "[e-mail]"}'

# Parse JSON into Pydantic model
user = User.parse_raw(json_data)
print(user)

# Invalid JSON (raises error)
# invalid_json = '{"username": "alice123", "email": 123}'  # Raises error

Explanation: Pydantic allows you to parse data directly from JSON using parse_raw(). If the data does not match the expected structure, it raises a validation error.

---

Key Concepts:

• BaseModel: All Pydantic models must inherit from BaseModel.

• validator: Custom field validation using the @validator decorator.

• @root_validator: Validation that operates on multiple fields at once.

• constr, conint, etc.: Field constraints for limiting values like minimum/maximum lengths, ranges, etc.

• Enums: Enforce a set of constant values for a field.

• Datetime Validation: Pydantic can automatically validate and parse datetime fields.

• JSON Parsing: Pydantic supports easy parsing from JSON data into models.

Pydantic makes it simple to define clear data models and apply validation rules, ensuring that data conforms to your expectations and improving data integrity in your application.