Image Processing

Image Processing in Intelligent Contracts lets contracts send screenshots, photos, or other visual data to LLMs alongside a prompt for analysis.

Sending Images to LLMs

Use the images parameter in gl.nondet.exec_prompt():

from genlayer import *
 
class ReceiptVerifier(gl.Contract):
    verified: bool
 
    def __init__(self):
        self.verified = False
 
    @gl.public.write
    def verify_receipt(self, image_data: bytes, expected_amount: str) -> None:
        def leader_fn():
            return gl.nondet.exec_prompt(
                f"Does this receipt show a payment of {expected_amount}? "
                "Respond as JSON: {{\"matches\": true/false, \"actual_amount\": \"...\"}}",
                images=[image_data],  # accepts raw bytes directly
                response_format="json",
            )
 
        def validator_fn(leaders_res) -> bool:
            if not isinstance(leaders_res, gl.vm.Return):
                return False
            my_result = leader_fn()
            return my_result["matches"] == leaders_res.calldata["matches"]
 
        result = gl.vm.run_nondet_unsafe(leader_fn, validator_fn)
        self.verified = result["matches"]

The images parameter accepts a sequence of raw bytes (e.g., PNG/JPEG data) or gl.nondet.Image objects.

Passing User-Supplied Images

Raw image bytes are convenient for small test fixtures and controlled inputs, but avoid pushing large image files directly through transaction calldata. Large payloads are harder to submit, review, and reproduce across environments. Prefer passing a stable external reference that validators can independently fetch.

A common pattern is:

Upload the image to a public HTTPS URL or a content-addressed location such as IPFS.
Pass the URL, and optionally a content hash, to the contract.
Fetch or render the URL inside the non-deterministic block.
Send the fetched image or screenshot to gl.nondet.exec_prompt(images=[...]).

import hashlib
 
from genlayer import *
 
class ImageClaimVerifier(gl.Contract):
    result: str
 
    def __init__(self):
        self.result = "pending"
 
    @gl.public.write
    def verify_claim_image(self, image_url: str, expected_sha256: str, claim: str) -> None:
        def leader_fn():
            res = gl.nondet.web.get(image_url)
            image_bytes = res.body
 
            if hashlib.sha256(image_bytes).hexdigest() != expected_sha256:
                raise gl.UserError("Image hash does not match the submitted reference")
 
            return gl.nondet.exec_prompt(
                f"Does this image support the claim: {claim}? "
                "Respond as JSON: {{\"supports_claim\": true/false}}",
                images=[image_bytes],
                response_format="json",
            )
 
        def validator_fn(leaders_res) -> bool:
            if not isinstance(leaders_res, gl.vm.Return):
                return False
            my_result = leader_fn()
            return my_result["supports_claim"] == leaders_res.calldata["supports_claim"]
 
        result = gl.vm.run_nondet_unsafe(leader_fn, validator_fn)
        self.result = "accepted" if result["supports_claim"] else "rejected"

For screenshots of web pages, use gl.nondet.web.render(url, mode='screenshot') instead of uploading screenshot bytes yourself. For user-uploaded files, keep the file off-chain and pass only the URL and hash on-chain.

Capturing Screenshots from the Web

Combine web access with image processing to screenshot a webpage and analyze it:

def check_website_status():
    url = "https://example.com/status-page"
    screenshot = gl.nondet.web.render(url, mode='screenshot')
    
    return gl.nondet.exec_prompt(
        "Is this status page showing all systems operational? "
        "Respond as JSON: {{\"all_operational\": true/false}}",
        images=[screenshot],
        response_format="json",
    )
 
result = gl.eq_principle.strict_eq(check_website_status)

Use Cases

Visual evidence verification — insurance claims with photo proof, damage assessment
Document analysis — receipts, invoices, certificates
Web monitoring — screenshot a page and verify its content matches expectations
Brand & content compliance — check if visual content meets guidelines
UI verification — screenshot an app and verify it renders correctly

Image processing requires vision-capable LLM models. On the GenLayer network, validators handle model selection — your contract just sends images. In Studio running locally, ensure your configured validators use a model that supports image inputs (e.g., GPT-5, Claude Sonnet).

Calling LLMs Web Access