{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## JSON Serialization\n", "\n", "In this notebook we will look how JSON can be used in the context of MaterialX interop.\n", "Items to examine include:\n", "\n", "* [Extracting out \"interfaces\" for `nodegraph`s](#nodegraph-interface-extraction)\n", "* [Converting XML to JSON.](#conversion-of-graphs-to-json)\n", "* [Determining a JSON schema.](#obtaining-a-json-schema-for-materialx)\n", "* [Validating JSON data against such as schema](#validating-materialx-json-with-schema)\n", "\n", "\n", "\n", "\n", "
\n", "\n", "
\n", "Snapshots of graphs generated using the JSONCrack add-on.\n", "\n", "The serialization logic has been added to utility Python command called `jsonIO.py`. The\n", "results of converting all of the files under the resources/Materials folder included with the MaterialX\n", "distribution can be found in the `JSON_outputs folder." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "### Setup for JSON support\n", "\n", "We will use the `xmltodict` Python package to convert from MaterialX represented in XML to JSON.\n", "The JSON package `json` will then be used to manipulate data." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "xmltodict version: 0.13.0\n", "json version: 2.0.9\n", "materialx version: 1.39.0\n" ] } ], "source": [ "import MaterialX as mx\n", "import mtlxutils.mxfile as mxf\n", "\n", "import xmltodict\n", "import json\n", "\n", "from IPython.display import display_markdown\n", "\n", "# Print xmltodict version\n", "print('xmltodict version: ', xmltodict.__version__)\n", "print('json version: ', json.__version__)\n", "print('materialx version: ', mx.__version__)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Nodegraph Interface Extraction \n", "\n", "We are interested in just the \"pattern graphs\" which are connected to any surface shader connected to a material.\n", "We define two utility functions:\n", "* `getShaderNodes()` : To get all the surface shader nodes either connected to a material or not.\n", "* `getRenderableGraphs` : To find any upstream graphs connected to a surface shader." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "def getShaderNodes(graphElement):\n", " '''\n", " Find all surface shaders in a GraphElement.\n", " '''\n", " shaderNodes = set()\n", " for material in graphElement.getMaterialNodes():\n", " for shader in mx.getShaderNodes(material):\n", " shaderNodes.add(shader.getNamePath())\n", " for shader in graphElement.getNodes():\n", " if shader.getType() == 'surfaceshader':\n", " shaderNodes.add(shader.getNamePath())\n", " return shaderNodes\n", "\n", "\n", "def getRenderableGraphs(graphElement):\n", " '''\n", " Find all renderable graphs in a GraphElement.\n", " '''\n", " ngnamepaths = set()\n", " graphs = []\n", " shaderNodes = getShaderNodes(graphElement)\n", " for shaderPath in shaderNodes:\n", " shader = doc.getDescendant(shaderPath)\n", " for input in shader.getInputs():\n", " ngString = input.getNodeGraphString()\n", " if ngString and ngString not in ngnamepaths:\n", " graphs.append(graphElement.getNodeGraph(ngString))\n", " ngnamepaths.add(ngString)\n", " return graphs" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "\n", "For each graph, we only wish to include the interface and include any connections on inputs or outputs. \n", "This provides a clean encapsulation of such a graph.\n", "\n", "All graphs are assumed to be parented directly under the root Document.\n", "\n", "Currently the process to just copy and extract this nodegraphs is overtly complex if we just want the interace. \n", "The `copyContentsFrom()` interface for nodegraphs copies over too much information requiring the removal of unneeded children.\n", "The alternative is to manually copy over nodegraph attributes, and child inputs and outputs.\n", "\n", "It would be useful to have the logic encapsulated in a single API call. \n", "\n", "We add in a `copyGraphInterfaces()` function which will copy only the interface of a `nodegraph` to a new `nodegraph` under a given document element. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def copyGraphInterfaces(dest, ng):\n", " '''\n", " Copy the interface of a nodegraph to a new nodegraph under a specified parent `dest`.\n", " '''\n", " copyMethod = 'add_remove'\n", " ng1 = dest.addNodeGraph(ng.getName())\n", " if copyMethod == 'add_remove':\n", " ng1.copyContentFrom(ng)\n", " for child in ng1.getChildren():\n", " if child.isA(mx.Input) or child.isA(mx.Output):\n", " for attr in ['nodegraph', 'nodename', 'defaultinput']:\n", " child.removeAttribute(attr)\n", " continue\n", " ng1.removeChild(child.getName())\n", " else:\n", " for attrName in ng.getAttributeNames():\n", " attr = ng.getAttribute(attrName)\n", " newattr = ng1.addAttribute(attr.getName(), attr.getType(), attr.getValue())\n", " newattr.copyContentFrom(attr)\n", " for port in ng.getInputs():\n", " newport = ng1.addInput(port.getName(), port.getType())\n", " newport.copyContentFrom(port)\n", " for port in ng.getOutputs():\n", " newport = ng1.addOutput(port.getName(), port.getType())\n", " newport.copyContentFrom(port)\n", " for attr in ['nodegraph', 'nodename', 'defaultinput']:\n", " newport.removeAttribute(attr) " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "The top level logic, loads in the \"Brick\" graph which can be found in the Examples folder of a MaterialX distribution.\n", "These are extracted out to a new document from which we get the representation as an XML string. \n", "The results are printed out. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
Extracted MaterialX Nodegraphs in XML\n", "\n", "```xml\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Load in sample MaterialX file\n", "doc, libFiles, status = mxf.MtlxFile.createWorkingDocument()\n", "mx.readFromXmlFile(doc, './data/standard_surface_brick_procedural.mtlx')\n", "\n", "# Create destination document and copy nodegraph interfaces over\n", "xmldoc = mx.createDocument()\n", "graphs = getRenderableGraphs(doc)\n", "for ng in graphs:\n", " copyGraphInterfaces(xmldoc, ng)\n", "\n", "# Get interfaces as an XML string\n", "xml_string = mxf.MtlxFile.writeDocumentToString(xmldoc)\n", "\n", "text = '
Extracted MaterialX Nodegraphs in XML\\n\\n' + '```xml\\n' + xml_string + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Test : XML text to JSON\n", "\n", "As a first test we will first just convert the XML string to JSON.\n", "\n", "After this extraction process we parse the `XML` string to produce the equivalent data as a `Python` dictionary using the `xmltodict.parse()` interfaces.\n", "\n", "This is then converted into a string to get the required `JSON` data using e `json.dumps`. \n", "( For the sake of display formatting some additional indentation is specified when dumping out a string. )" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
MaterialX in JSON\n", "\n", "```json\n", "{\n", " \"materialx\": {\n", " \"version\": \"1.39\",\n", " \"nodegraph\": {\n", " \"name\": \"NG_BrickPattern\",\n", " \"input\": [\n", " {\n", " \"name\": \"brick_color\",\n", " \"type\": \"color3\",\n", " \"value\": \"0.661876, 0.19088, 0\",\n", " \"uiname\": \"Brick Color\",\n", " \"uifolder\": \"Color\"\n", " },\n", " {\n", " \"name\": \"hue_variation\",\n", " \"type\": \"float\",\n", " \"value\": \"0.083\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Hue Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " {\n", " \"name\": \"value_variation\",\n", " \"type\": \"float\",\n", " \"value\": \"0.787\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Value Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " {\n", " \"name\": \"roughness_amount\",\n", " \"type\": \"float\",\n", " \"value\": \"0.853\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Roughness Amount\",\n", " \"uifolder\": \"Roughness\"\n", " },\n", " {\n", " \"name\": \"dirt_color\",\n", " \"type\": \"color3\",\n", " \"value\": \"0.56372, 0.56372, 0.56372\",\n", " \"uiname\": \"Dirt Color\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " {\n", " \"name\": \"dirt_amount\",\n", " \"type\": \"float\",\n", " \"value\": \"0.248\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Dirt Amount\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " {\n", " \"name\": \"uvtiling\",\n", " \"type\": \"float\",\n", " \"value\": \"3\",\n", " \"uisoftmin\": \"1\",\n", " \"uisoftmax\": \"16\",\n", " \"uiname\": \"UV Tiling\",\n", " \"uifolder\": \"Texturing\"\n", " }\n", " ],\n", " \"output\": [\n", " {\n", " \"name\": \"base_color_output\",\n", " \"type\": \"color3\"\n", " },\n", " {\n", " \"name\": \"specular_roughness_output\",\n", " \"type\": \"float\"\n", " },\n", " {\n", " \"name\": \"normal_output\",\n", " \"type\": \"vector3\"\n", " }\n", " ]\n", " }\n", " }\n", "}\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Build JSON from XML\n", "options = {\n", " 'attr_prefix': '', # Set prefix for attributes\n", "}\n", "python_dict = xmltodict.parse(xml_string, **options)\n", "json_string = json.dumps(python_dict)\n", "json_string_fmt = json.dumps(python_dict, indent=2)\n", "\n", "text = '
MaterialX in JSON\\n\\n' + '```json\\n' + json_string_fmt + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)\n", "\n", "with open('mtlx_brick.json', 'w') as jsonfile:\n", " jsonfile.write(json_string_fmt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Explicit Conversion\n", "\n", "The desire is to introduce a standardized JSON representation for MaterialX. For this a match for what is supported for XML is required.\n", "\n", "XML is supported via the `MaterialXFormat` library. This this JSON can be added.\n", "\n", "For this book, we will create a bidirectional conversion between XML and JSON. Thus instead of \n", "a non-standard conversion using a generic converter like `xmltodict` we will the logic in these scripts (and eventually \n", "the `MaterialXFormat` library) for JSON conversion.\n", "\n", "Some key factors to consider include:\n", "1. The JSON representation should be a direct match to the XML representation.\n", "2. The JSON representation should match the XML size as closely as possible.\n", "3. Export and import options for XML should be supported for JSON.\n", "4. There is no concept of \"includes\" in JSON. This is a concept specific to XML." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### JSON Serialization\n", "\n", "To perform a proper serialization, the MaterialX document itself should be examined with direct conversion to JSON. \n", "\n", "##### Serialization to JSON\n", "\n", "For conversion to JSON we introduce two functions:\n", "\n", "1. `documentToJSON()` : Converts a MaterialX document to a JSON string.\n", "2. `elementToJSON()` : Converts a MaterialX element to a JSON string, and continues to recursively convert any children." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# We use a colon to separate the category and name of an element in the JSON hierarchy\n", "JSON_CATEGORY_NAME_SEPARATOR = ':'\n", "# The root of the JSON hierarchy\n", "MATERIALX_DOCUMENT_ROOT = 'materialx'\n", "\n", "# Convert MaterialX element to JSON\n", "def elementToJSON(elem, jsonParent):\n", " '''\n", " Convert an MaterialX XML element to JSON.\n", " Will recursively traverse the parent/child Element hierarchy.\n", " '''\n", " if (elem.getSourceUri() != \"\"):\n", " return\n", " \n", " # Create a new JSON element for the MaterialX element\n", " jsonElem = {}\n", "\n", " # Add attributes\n", " for attrName in elem.getAttributeNames():\n", " jsonElem[attrName] = elem.getAttribute(attrName)\n", "\n", " # Add children\n", " for child in elem.getChildren():\n", " jsonElem = elementToJSON(child, jsonElem)\n", " \n", " # Add element to parent\n", " jsonParent[elem.getCategory() + JSON_CATEGORY_NAME_SEPARATOR + elem.getName()] = jsonElem\n", " return jsonParent\n", "\n", "# Convert MaterialX document to JSON\n", "def documentToJSON(doc):\n", " '''Convert an MaterialX XML document to JSON'''\n", " root = {}\n", " root[\"materialx\"] = {}\n", "\n", " for attrName in doc.getAttributeNames():\n", " root[attrName] = doc.getAttribute(attrName)\n", "\n", " for elem in doc.getChildren():\n", " elementToJSON(elem, root[MATERIALX_DOCUMENT_ROOT])\n", "\n", " result = json.dumps(root, indent=2)\n", " return result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We call `documentToJSON()` to convert both the entire document as well as just the NodeGraph interface document below:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
Entire document to JSON\n", "\n", "```json\n", "{\n", " \"materialx\": {\n", " \"nodegraph:NG_BrickPattern\": {\n", " \"input:brick_color\": {\n", " \"type\": \"color3\",\n", " \"value\": \"0.661876, 0.19088, 0\",\n", " \"uiname\": \"Brick Color\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:hue_variation\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.083\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Hue Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:value_variation\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.787\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Value Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:roughness_amount\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.853\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Roughness Amount\",\n", " \"uifolder\": \"Roughness\"\n", " },\n", " \"input:dirt_color\": {\n", " \"type\": \"color3\",\n", " \"value\": \"0.56372, 0.56372, 0.56372\",\n", " \"uiname\": \"Dirt Color\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " \"input:dirt_amount\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.248\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Dirt Amount\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"float\",\n", " \"value\": \"3\",\n", " \"uisoftmin\": \"1\",\n", " \"uisoftmax\": \"16\",\n", " \"uiname\": \"UV Tiling\",\n", " \"uifolder\": \"Texturing\"\n", " },\n", " \"multiply:node_multiply_5\": {\n", " \"type\": \"color3\",\n", " \"input:in1\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_mix_6\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_7\"\n", " }\n", " },\n", " \"mix:node_mix_8\": {\n", " \"type\": \"color3\",\n", " \"input:fg\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_multiply_5\"\n", " },\n", " \"input:bg\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_multiply_9\"\n", " },\n", " \"input:mix\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_10\"\n", " }\n", " },\n", " \"constant:node_color_11\": {\n", " \"type\": \"color3\",\n", " \"input:value\": {\n", " \"type\": \"color3\",\n", " \"value\": \"0.263273, 0.263273, 0.263273\"\n", " }\n", " },\n", " \"multiply:node_multiply_9\": {\n", " \"type\": \"color3\",\n", " \"input:in1\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_color_11\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_7\"\n", " }\n", " },\n", " \"rgbtohsv:node_rgbtohsv_12\": {\n", " \"type\": \"color3\",\n", " \"input:in\": {\n", " \"type\": \"color3\",\n", " \"interfacename\": \"brick_color\"\n", " }\n", " },\n", " \"combine3:node_combine3_color3_13\": {\n", " \"type\": \"color3\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_14\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"value\": \"0\"\n", " },\n", " \"input:in3\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_15\"\n", " }\n", " },\n", " \"add:node_add_16\": {\n", " \"type\": \"color3\",\n", " \"input:in1\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_combine3_color3_13\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_rgbtohsv_12\"\n", " }\n", " },\n", " \"hsvtorgb:node_hsvtorgb_17\": {\n", " \"type\": \"color3\",\n", " \"input:in\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_add_16\"\n", " }\n", " },\n", " \"subtract:node_subtract_18\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_add_19\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.35\"\n", " }\n", " },\n", " \"multiply:node_multiply_14\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_subtract_18\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"hue_variation\"\n", " }\n", " },\n", " \"multiply:node_multiply_15\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_add_19\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_20\"\n", " }\n", " },\n", " \"clamp:node_clamp_0\": {\n", " \"type\": \"color3\",\n", " \"input:in\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_mix_8\"\n", " }\n", " },\n", " \"multiply:node_multiply_1\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_divide_21\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_22\"\n", " }\n", " },\n", " \"max:node_max_1\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_10\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.00001\"\n", " }\n", " },\n", " \"divide:node_divide_21\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"roughness_amount\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_max_1\"\n", " }\n", " },\n", " \"mix:node_mix_6\": {\n", " \"type\": \"color3\",\n", " \"input:fg\": {\n", " \"type\": \"color3\",\n", " \"interfacename\": \"dirt_color\"\n", " },\n", " \"input:bg\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_hsvtorgb_17\"\n", " },\n", " \"input:mix\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_23\"\n", " }\n", " },\n", " \"multiply:node_multiply_23\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"dirt_amount\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_24\"\n", " }\n", " },\n", " \"multiply:node_multiply_25\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"hue_variation\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_26\"\n", " }\n", " },\n", " \"add:node_add_19\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_25\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_7\"\n", " }\n", " },\n", " \"multiply:node_multiply_20\": {\n", " \"type\": \"float\",\n", " \"input:in1\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"value_variation\"\n", " },\n", " \"input:in2\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_tiledimage_float_26\"\n", " }\n", " },\n", " \"normalmap:node_normalmap_3\": {\n", " \"type\": \"vector3\",\n", " \"input:in\": {\n", " \"type\": \"vector3\",\n", " \"nodename\": \"node_tiledimage_vector3_27\"\n", " }\n", " },\n", " \"convert:node_convert_1\": {\n", " \"type\": \"vector2\",\n", " \"input:in\": {\n", " \"type\": \"float\",\n", " \"interfacename\": \"uvtiling\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_vector3_27\": {\n", " \"type\": \"vector3\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_normal.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_float_22\": {\n", " \"type\": \"float\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_roughness.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_float_10\": {\n", " \"type\": \"float\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_mask.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_float_7\": {\n", " \"type\": \"float\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_base_gray.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_float_26\": {\n", " \"type\": \"float\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_variation_mask.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"tiledimage:node_tiledimage_float_24\": {\n", " \"type\": \"float\",\n", " \"input:file\": {\n", " \"type\": \"filename\",\n", " \"value\": \"brick_dirt_mask.jpg\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"vector2\",\n", " \"nodename\": \"node_convert_1\"\n", " }\n", " },\n", " \"output:base_color_output\": {\n", " \"type\": \"color3\",\n", " \"nodename\": \"node_clamp_0\"\n", " },\n", " \"output:specular_roughness_output\": {\n", " \"type\": \"float\",\n", " \"nodename\": \"node_multiply_1\"\n", " },\n", " \"output:normal_output\": {\n", " \"type\": \"vector3\",\n", " \"nodename\": \"node_normalmap_3\"\n", " }\n", " },\n", " \"standard_surface:N_StandardSurface\": {\n", " \"type\": \"surfaceshader\",\n", " \"input:base_color\": {\n", " \"type\": \"color3\",\n", " \"nodegraph\": \"NG_BrickPattern\",\n", " \"output\": \"base_color_output\"\n", " },\n", " \"input:specular_roughness\": {\n", " \"type\": \"float\",\n", " \"nodegraph\": \"NG_BrickPattern\",\n", " \"output\": \"specular_roughness_output\"\n", " },\n", " \"input:normal\": {\n", " \"type\": \"vector3\",\n", " \"nodegraph\": \"NG_BrickPattern\",\n", " \"output\": \"normal_output\"\n", " }\n", " },\n", " \"surfacematerial:M_BrickPattern\": {\n", " \"type\": \"material\",\n", " \"input:surfaceshader\": {\n", " \"type\": \"surfaceshader\",\n", " \"nodename\": \"N_StandardSurface\"\n", " }\n", " }\n", " },\n", " \"version\": \"1.39\",\n", " \"colorspace\": \"lin_rec709\",\n", " \"fileprefix\": \"../../../Images/\"\n", "}\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/markdown": [ "
Node Graph Interface to JSON\n", "\n", "```json\n", "{\n", " \"materialx\": {\n", " \"nodegraph:NG_BrickPattern\": {\n", " \"input:brick_color\": {\n", " \"type\": \"color3\",\n", " \"value\": \"0.661876, 0.19088, 0\",\n", " \"uiname\": \"Brick Color\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:hue_variation\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.083\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Hue Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:value_variation\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.787\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Value Variation\",\n", " \"uifolder\": \"Color\"\n", " },\n", " \"input:roughness_amount\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.853\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Roughness Amount\",\n", " \"uifolder\": \"Roughness\"\n", " },\n", " \"input:dirt_color\": {\n", " \"type\": \"color3\",\n", " \"value\": \"0.56372, 0.56372, 0.56372\",\n", " \"uiname\": \"Dirt Color\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " \"input:dirt_amount\": {\n", " \"type\": \"float\",\n", " \"value\": \"0.248\",\n", " \"uimin\": \"0\",\n", " \"uimax\": \"1\",\n", " \"uiname\": \"Dirt Amount\",\n", " \"uifolder\": \"Dirt\"\n", " },\n", " \"input:uvtiling\": {\n", " \"type\": \"float\",\n", " \"value\": \"3\",\n", " \"uisoftmin\": \"1\",\n", " \"uisoftmax\": \"16\",\n", " \"uiname\": \"UV Tiling\",\n", " \"uifolder\": \"Texturing\"\n", " },\n", " \"output:base_color_output\": {\n", " \"type\": \"color3\"\n", " },\n", " \"output:specular_roughness_output\": {\n", " \"type\": \"float\"\n", " },\n", " \"output:normal_output\": {\n", " \"type\": \"vector3\"\n", " }\n", " }\n", " },\n", " \"version\": \"1.39\"\n", "}\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Convert entire document\n", "doc_result = documentToJSON(doc)\n", "\n", "text = '
Entire document to JSON\\n\\n' + '```json\\n' + doc_result + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)\n", "\n", "# Convert just the graph\n", "graph_result = documentToJSON(xmldoc)\n", "\n", "text = '
Node Graph Interface to JSON\\n\\n' + '```json\\n' + graph_result + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The JSON is visualized as a graph for the entire document below, as well as another sample conversion of the glTF \"Boombox\" and \"Olvies\" examples from the sample models library\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Graph of Marble Material\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Graph of \"Boombox\" Example\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Graph of \"Olives\" Example\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Deserialization from JSON\n", "\n", "For conversion from JSON we introduce two functions:\n", "\n", "1. `documentFromJSON()` : Converts a JSON string to a MaterialX document.\n", "2. `elementFromJSON()` : Converts a JSON string to a MaterialX element, and continues to recursively convert any children.\n", "\n", "Note that to perform deserialization we need to split the `category` and `name` out for non-attribute elements." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Separator between category and name in JSON element\n", "JSON_CATEGORY_NAME_SEPARATOR = \":\"\n", "\n", "# Convert JSON element to MaterialX\n", "def elementFromJSON(node, elem):\n", " '''\n", " Convert an JSON element to MaterialX\n", " '''\n", " for key in node:\n", " value = node[key]\n", "\n", " # Set attributes \n", " if isinstance(value, str):\n", " elem.setAttribute(key, str(value))\n", "\n", " # Traverse chilren\n", " else:\n", " # Traverse down from root\n", " if key == MATERIALX_DOCUMENT_ROOT:\n", " elementFromJSON(value, elem)\n", " continue\n", "\n", " # Split key name by \":\" to get category and name\n", " category, name = key.split(JSON_CATEGORY_NAME_SEPARATOR, 1)\n", " if category and not elem.getChild(name):\n", " child = elem.addChildOfCategory(category, name)\n", " elementFromJSON(value, child)\n", "\n", "# Convert JSON to MaterialX document\n", "def documentFromJSON(jsonDoc, doc):\n", " '''\n", " Convert a JSON document to MaterialX\n", " '''\n", " elementFromJSON(jsonDoc, doc)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using these functions we can convert the JSON document and NodeGraph results back to a MaterialX documents.\n", "\n", "We also add in validation and an \"upgrade\" call to roughly match what is performed for XML serialization." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
JSON Deserialization of Document\n", "\n", "```xml\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Convert entire document back from JSON\n", "newDoc = mx.createDocument() \n", "jsonObject = json.loads(doc_result)\n", "documentFromJSON(jsonObject, newDoc)\n", "\n", "# Validate and upgrade element version\n", "valid, errors = newDoc.validate()\n", "if not valid:\n", " print('Validation errors:')\n", " for err in errors:\n", " print(' {}'.format(err))\n", "newDoc.upgradeVersion()\n", "\n", "newDocString = mx.writeToXmlString(newDoc) \n", "text = '
JSON Deserialization of Document\\n\\n' + '```xml\\n' + newDocString + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
JSON Deserialization of NodeGraph\n", "\n", "```xml\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Convert nodegraph interface back from JSON\n", "newDoc = mx.createDocument() \n", "jsonObject = json.loads(graph_result)\n", "documentFromJSON(jsonObject, newDoc)\n", "\n", "# Validate and upgrade element version\n", "valid, errors = newDoc.validate()\n", "if not valid:\n", " print('Validation errors:')\n", " for err in errors:\n", " print(' {}'.format(err))\n", "newDoc.upgradeVersion()\n", "\n", "newDocString = mx.writeToXmlString(newDoc) \n", "text = '
JSON Deserialization of NodeGraph\\n\\n' + '```xml\\n' + newDocString + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Obtaining a JSON Schema for MaterialX \n", "\n", "For this example we will use the first test results just the `NodeGraph`` for simplicity. It is possible to create a schema for all elements of MaterialX which would include all the definitions which are part of the standard library.\n", "\n", "The schema is created using the [OpenAI](https://platform.openai.com/docs/api-reference) Python package. Various MaterialX documents (saved out in JSON) were used as input data.\n", "\n", "A graph of the schema is shown below, with the textual description below:\n", "\n", "\n", "Some examples and test suite documents are suitable to obtain most of the schema. Small edits we're made for anything amiss such setting \"required\" attributes.\n", "\n", "Note that this is just a sample schema. It is not complete. It is also not the only possible schema." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
MaterialX JSON Schema\n", "\n", "```json\n", "{\n", " \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n", " \"definitions\": {\n", " \"port\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"name\": {\n", " \"type\": \"string\"\n", " },\n", " \"type\": {\n", " \"type\": \"string\",\n", " \"enum\": [\n", " \"float\",\n", " \"vector2\",\n", " \"vector3\",\n", " \"vector4\",\n", " \"color3\",\n", " \"color4\",\n", " \"bool\",\n", " \"integer\",\n", " \"filename\"\n", " ]\n", " },\n", " \"nodename\": {\n", " \"type\": \"string\"\n", " }\n", " },\n", " \"required\": [\n", " \"name\",\n", " \"type\"\n", " ]\n", " },\n", " \"inputPort\": {\n", " \"allOf\": [\n", " {\n", " \"$ref\": \"#/definitions/port\"\n", " },\n", " {\n", " \"properties\": {\n", " \"value\": {\n", " \"type\": \"string\"\n", " },\n", " \"uiname\": {\n", " \"type\": \"string\"\n", " },\n", " \"uifolder\": {\n", " \"type\": \"string\"\n", " },\n", " \"uimin\": {\n", " \"type\": \"string\"\n", " },\n", " \"uimax\": {\n", " \"type\": \"string\"\n", " },\n", " \"uisoftmin\": {\n", " \"type\": \"string\"\n", " },\n", " \"uisoftmax\": {\n", " \"type\": \"string\"\n", " }\n", " },\n", " \"required\": [\n", " \"value\"\n", " ]\n", " }\n", " ]\n", " },\n", " \"outputPort\": {\n", " \"allOf\": [\n", " {\n", " \"$ref\": \"#/definitions/port\"\n", " }\n", " ]\n", " },\n", " \"nodegraph\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"name\": {\n", " \"type\": \"string\"\n", " },\n", " \"input\": {\n", " \"type\": \"array\",\n", " \"items\": {\n", " \"$ref\": \"#/definitions/inputPort\"\n", " }\n", " },\n", " \"output\": {\n", " \"type\": \"array\",\n", " \"items\": {\n", " \"$ref\": \"#/definitions/outputPort\"\n", " }\n", " }\n", " },\n", " \"required\": [\n", " \"name\",\n", " \"input\",\n", " \"output\"\n", " ]\n", " },\n", " \"materialx\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"@version\": {\n", " \"type\": \"string\"\n", " },\n", " \"nodegraph\": {\n", " \"$ref\": \"#/definitions/nodegraph\"\n", " }\n", " },\n", " \"required\": [\n", " \"version\",\n", " \"nodegraph\"\n", " ]\n", " }\n", " },\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"materialx\": {\n", " \"$ref\": \"#/definitions/materialx\"\n", " }\n", " },\n", " \"required\": [\n", " \"materialx\"\n", " ]\n", "}\n", "```\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Define the JSON schema. \n", "schema = {\n", " \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n", " \"definitions\": {\n", " \"port\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"name\": {\n", " \"type\": \"string\"\n", " },\n", " \"type\": {\n", " \"type\": \"string\",\n", " \"enum\": [\"float\", \"vector2\", \"vector3\", \"vector4\", \"color3\", \"color4\", \"bool\", \"integer\", \"filename\"]\n", " },\n", " \"nodename\": {\n", " \"type\": \"string\"\n", " }\n", " },\n", " \"required\": [\"name\", \"type\"]\n", " },\n", " \"inputPort\": {\n", " \"allOf\": [\n", " {\n", " \"$ref\": \"#/definitions/port\"\n", " },\n", " {\n", " \"properties\": {\n", " \"value\": {\n", " \"type\" : \"string\"\n", " },\n", " \"uiname\": {\n", " \"type\": \"string\"\n", " },\n", " \"uifolder\": {\n", " \"type\": \"string\"\n", " },\n", " \"uimin\": {\n", " \"type\": \"string\"\n", " },\n", " \"uimax\": {\n", " \"type\": \"string\"\n", " },\n", " \"uisoftmin\": {\n", " \"type\": \"string\"\n", " },\n", " \"uisoftmax\": {\n", " \"type\": \"string\"\n", " }\n", " },\n", " \"required\": [\"value\"]\n", " }\n", " ]\n", " },\n", " \"outputPort\": {\n", " \"allOf\": [\n", " {\n", " \"$ref\": \"#/definitions/port\"\n", " }\n", " ]\n", " },\n", " \"nodegraph\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"name\": {\n", " \"type\": \"string\"\n", " },\n", " \"input\": {\n", " \"type\": \"array\",\n", " \"items\": {\n", " \"$ref\": \"#/definitions/inputPort\"\n", " }\n", " },\n", " \"output\": {\n", " \"type\": \"array\",\n", " \"items\": {\n", " \"$ref\": \"#/definitions/outputPort\"\n", " }\n", " }\n", " },\n", " \"required\": [\"name\", \"input\", \"output\"]\n", " },\n", " \"materialx\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"@version\": {\n", " \"type\": \"string\"\n", " },\n", " \"nodegraph\": {\n", " \"$ref\": \"#/definitions/nodegraph\"\n", " }\n", " },\n", " \"required\": [\"version\", \"nodegraph\"]\n", " }\n", " },\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"materialx\": {\n", " \"$ref\": \"#/definitions/materialx\"\n", " }\n", " },\n", " \"required\": [\"materialx\"]\n", "}\n", "\n", "# Write the schema to a file\n", "jsonSchemaFilePath = 'data/mtlx_sample_schema.json'\n", "with open(jsonSchemaFilePath, 'w') as f:\n", " json.dump(schema, f)\n", "\n", "# Read the schema from a file\n", "loaded_schema = {}\n", "with open(jsonSchemaFilePath, 'r') as schema_file:\n", " loaded_schema = json.loads(schema_file.read())\n", "\n", "text = '
MaterialX JSON Schema\\n\\n' + '```json\\n' + json.dumps(schema, indent=2) + '\\n```\\n' + '
\\n' \n", "display_markdown(text, raw=True)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Validating MaterialX JSON with Schema \n", "\n", "To perform validation we will use the `validate` interface from the [`jsconschema`](https://pypi.org/project/jsonschema/) package." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "JSON string is valid against the schema\n" ] } ], "source": [ "# Validate JSON formatted MaterialX document.\n", "from jsonschema import validate as jsvalidate\n", "\n", "# Validate the JSON string against the schema\n", "def validateJson(json_string, schema):\n", " try:\n", " data = json.loads(json_string)\n", " jsvalidate(data, schema)\n", " except Exception as e:\n", " return e\n", " return ''\n", "\n", "\n", "error = validateJson(json_string, loaded_schema)\n", "if not error:\n", " print(\"JSON string is valid against the schema\")\n", "else:\n", " print('JSON string is not valid:\\n')\n", " print(error)" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.10" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }