ns_xml 2.0 Requirements and Design

Motivation

ns_xml as it exists currently is an integral part of OpenACS 4.x, but its implementation fails to expose several important aspects of libxml's tree manipulation API. As a result, it is impossible to create some legal XML documents using ns_xml. The shortcoming lies in ns_xml's inability to create node content through the use of text nodes. In ns_xml, the only way to manipulate the content of a node is to either initialize the node's content at creation time (e.g. through a node new_child call) or by setting it explicitly through node setcontent. That overlooks the case of this simple XML snippet:

<sentence>Here is <keyword>my</keyword> sentence</sentence>

That is only part of the challenge, however. ns_xml is also limited to the linear creation of a document. This is great if your goal is to programmatically serialize data in a database, which is the obvious use of ns_xml within OpenACS. If you hope to mutate the structure of an existing document or create a document in a random-access fashion (e.g. through a user interface), however, you're in trouble. The following shortcomings exist:

• Nodes can not be inserted before other existing nodes at the same level.
• Nodes can not be deleted.
• Nodes can not be moved.
• Nodes can not be copied.
• You can't find the parent of a node, so tree traversal is one-way (top-down).
• An attribute can not be unset. It can be set to null, but not made to disappear.

The Requirements

1. Support for all legacy (ns_xml 1.x) Tcl code.
2. Support for text node creation.
3. Support for "insert as previous" node creation for both text and element nodes.
4. Support for node deletion (recursive).
5. Support for node relocation, including cross-document relocation. Nodes moved between documents of different persistence states will inherit the persistence of the new document (i.e. a node moved from a persistent document to a transient document will vanish at the end of the Tcl interpreter's session).
6. Support for in-place node duplication (recursive). The duplicate will be instantiated as the sibling following the cloned node.
7. Support for bottom-up tree traversal, from child to parent.
8. Support for unsetting attributes

1. Support rendering of any node as a stand-alone XML document (recursive)
2. Get rid of the ns_xml doc free naming convention. This is Tcl. We don't want to think about freeing memory. We're just deleting persistent documents here. :)

The Design

Creating this functionality means adding several commands to the ns_xml API. The naming scheme of ns_xml's existing calls is very intuitive to the Tcl developer. However, it quickly became apparent that the naming scheme wasn't general enough to encompass the new functionality cleanly. If an attempt were made to add the functionality described above within the existing naming scheme, all transparency would be lost. Tcl developers would find themselves referring to reference material constantly. Luckily, the naming scheme proposed below does not conflict with the existing scheme (except in cases where the commands are identical between the two schemes). Thus, all the deprecated calls can be mapped to their 2.0 equivalents, and a user can run legacy Tcl code unmodified.

In the new scheme, commands are divided into four functional buckets which largely coincide with their first word. All calls follow the grammatical convention:

subject verb [object]

Note that the Node Interaction bucket is subdivided into several categories due to its complexity.

The 2.0 API

• Document Instantiation
• set xml_doc_id [ns_xml string parse xml ?-persist? ?-validate? string]
• set xsl_doc_id [ns_xml string parse xsl ?-persist? ?-validate? string]
• set xml_doc_id [ns_xml create xml ?-persist? ?doc-version?]
• Document Transformation
• set new_xml_doc_id [ns_xml transform ?-persist? xml_doc_id xsl_doc_id]
• Document Interaction
• set node_id [ns_xml doc get root doc_id]
• set node_id [ns_xml doc create root doc_id node_name node_content]
• set node_id [ns_xml doc render doc_id]
• set node_id [ns_xml doc delete doc_id]
• set node_id [ns_xml doc cleanup doc_id] (tolerant of already deleted documents)
• Node Interaction
• Tree Traversal
• set node_id_list [ns_xml node get children node_id]
• set node_id [ns_xml node get parent node_id]
• Node Instantiation
• set node_id [ns_xml node create child_node node_id name content]
• set node_id [ns_xml node create child_text node_idcontent]
• set node_id [ns_xml node create prev_sibling_node node_id name content]
• set node_id [ns_xml node create prev_sibling_text node_id content]
• set node_id [ns_xml node create next_sibling_node node_id name content]
• set node_id [ns_xml node create next_sibling_text node_idcontent]
• Node Duplication
• new_node_id [ns_xml node clone node_id]
• Node Relocation - note that relinked nodes receive new node_ids. The old ids become defunct.
• set new_node_id [ns_xml node relink_as child node_id new_parent_node_id]
• set new_node_id [ns_xml node relink_as prev_sibling node_id new_sibling_node_id]
• set new_node_id [ns_xml node relink_as next_sibling node_id new_prev_sibling_node_id]
• Node Removal
• ns_xml node delete node_id
• Value Retrieval
• set string [ns_xml node get attr node_id prop_name]
• set string [ns_xml node get name node_id prop_name]
• set string [ns_xml node get type node_id prop_name]
• set string [ns_xml node get content node_id prop_name]
• Value Manipulation
• ns_xml node set attr node_id prop_name string
• ns_xml node unset attr node_id prop_name
• ns_xml node set content node_id string
• Node Serialization
• set string [ns_xml node render node_id]

Why'd you do that? (a.k.a. Design Considerations)

• Why ns_xml string parse and not just ns_xml parse?

  Because future support may be added for parsing filehandles in addition to tcl in-memory strings.

• Why ns_xml create xml and not just ns_xml create?

  This leaves the door open for dynamic creation of other documents like XSL or DTDs in the future. Note that XSL can currently only be parsed and applied to XML documents as-is.

• Why implement both ns_xml node create child and ns_xml node create next_sibling when you can just create a new child of a given node's parent? (or similar questions)

  This API walks a fine line between developer usability and cleanliness. The number of calls could have been drastically reduced as well if we offered a lower-level API requiring developers to first instantiate nodes and then link them as they chose. But that makes the developer's life harder (not to mention decreasing performance by putting code that would be in C in Tcl), which is what we're trying to avoid. We're also trying to get a fair amount of interoperability with the 1.x API, whose lack of a get parent command mandated the use of the create next_sibling command in cases where the parent was inconvenient or impossible to derive. So deprecating the whole create sibling concept seemed a bit harsh, especially since it's speedier than the Tcl workaround.

• Why does the XSL transformation command have the opposite arg order to the old one?

  Because the old ordering goes against every other command in the API

• Why is there an xml node render command? You could clone the node, relink the copy to a new document and render that!

  Partly because it creates parallelism with the document itself in that a node can be created, futzed with, serialized and deleted. Partly because it's more efficient than the alternative. Partly because it's useful in creating a nice UI for editing complicated XML documents. Partly because I say so. ;-)