# jsonpath IETF Working Group May 2021 Interim Date: 2021-05-11 Time: 9:00 - 11:00 UTC * Chair: Tim Bray * Chair: James Gruessing * Area Director: Francesca Palombini * Scribe: James Gruessing, Carsten Bormann # Attendees 1. Carsten Bormann, TZI 2. # Agenda Bashing None # Discussion ## Terminology - #84 Tim: On email you said we are good on the terminology issue Stefan: We need to some work and finish in quite a short time Tim: If I understand the state of the discussion, we should stick to 8259 terminology where possible, and introduce terms as necessary Everything is a JSON value, except member names and reasonable to have something shorter, candidates such as "node" Calling for opinions on this? Carsten: We don't have to start from January, we have a few things in the document and a few open issues for indexing filters and syntactic elements We have had "indexing", "key" for the first thing There are a few details we have to understand about selectors and grouping Carsten: Maybe we need a term for things with indexing, including all three kinds Maybe it's not helpful to separate the kinds into different categories Greg: The selector is the operator, the encompassing thing Index is the bracket selector, indexes are the things in there The selector is broad, the path is a sequence of selectors Carsten: I think we have consensus on overall structure of this Stefan: We need a specific term for index-value pair Glyn: I have a concern about the use of "index", it carries array connotations Greg: I see where you're coming from, having the understanding that any data type can be an index makes it more a "general use term" Tim: "key" has object connotation, I'm okay with either of these I'm not hearing a strong objection to the alternatives Marko: This summary doesn't capture array slice selector is an index, which can be repeated The slice expression is a valid way to select things Carsten: Good idea to make a union a comma-separated list of slices without weakening the term index Marko: The index cannot be expressed just as generic form as it's not a valid start value Carsten: There is something to solve at the union selector Marko: The union selector is index or key, filtered Greg: A filter selector would also work Tim: I'm not hearing strong issues of principle here, but an editorial issue and this should be handed to the editors Carsten: We should give this to the editors to handle, but have a leeway, let's agree direction Greg: Supposing we call an index a number, the union is a comma-delimited sequence or filters, or slices. Carsten: You're making a technical change Marko: Is a wildcard a special case? Stefan: We are looking for a term for this index value pair I think Greg: The difference with this terminology is we're not talking about values, only the value or numerical index Stefan: If we take the normalised expression, brackets are always keys until we find the final value Stefan: If we call it "index" I'm fine with it, but if Glyn says index is more connected to array Glyn: I am happy to wait for a PR Carsten: I'm happy to do it Tim: I'm not detecting any disagreement on this Greg: I'll write the terminology PR Tim: Are we still sticking with the term "node"? Greg: It is the pairing of the location and the value **Consensus**: Sticking with the term "node" implying the location and the value **Action**: Greg to PR the terminology ## Relative Path Support - #59 Tim: I think the use of relative paths is not controversial, but the issue is using '@' Glyn: I thought it was using @ outside the filter expression? Marko: To supply a dynamic value to an index? Glyn: The use case is an "escape hatch" to provide current value to the processor Carsten: Do we have an example? Glyn: David provides an example in the issue but I don't favour it Stefan: If you allow query expressions with an @ symbol that part much be defined elsewhere Greg: That is David's point, and that some external tool that has navigated partially into an input, and wants to navigate relatively Stefan: If you define a relative path, will you always need the root path? If not, it would suffice to redefine the root Greg: Are you suggesting getting rid of the $ and just using @? Stefan: Just using $ and don't talk about @ Carsten: If there's a difference between notating $ and @, if there is no difference we can always use $ Greg: There's a distinct difference wanting relative and root On March 12th I give an example where both are used, in some cases back to root, in other to specify the local Marko: What I was trying to write is the difference; we could just re-evaluate with the current node Where we wouldn't want to do that, is when we need to go back above the current root in a filter expression or such Stefan: There should be consensus about the fact we cannot go back Greg: There a data models that don't support navigating to the parent of a value Marko: It's not about having navigation, it's about having two handles Carsten: I can imagine a use case where we need to address this Tim: There's an argument to be made this is outside our charter, of known implementations 25% support them Greg: Mine does not We could say the spec says "start with the $", and optionally support starting with an @ Tim: I hate optional behaviours in standards due to interoperability This smells outside our charter Carsten: We need to define our extensibility model before we finish this issue One way is to nail down the feature set once, everything not in that feature set is not valid Another way is to introduce versions, with linear progression Both are extremely unattractive, so we should group optional things into small number of features Greg: Another option is to just allow and support $, and fail if it doesn't start with $ Glyn: I think this is outside our agenda Greg: I'm saying we don't move forward with this, but if implementations use it in future we decide to update the spec we can include it Stefan: A very cheap thing might be is to always start the query with the $, but the $ addresses not just the root Tim: It's not obvious that relative path and @ falls into the "common semantics" and other aspects of the existing implementations Carsten: This does not relieve us of the extensibility model Glyn: If we make @ at the beginning of a path a syntax error, one model is to allow implementations to override syntax errors Tim: I haven't seen a specific proposal, Carsten did I miss something? Carsten: I haven't written this up yet, but we should apply existing IETF experience here We should identify extension points so it can be done in an orderly way Tim: One extension point might be the first character of a query? Carsten: Yes, and an IANA registry of first characters with a registry policy We should have a defined model for implementors or this WG to define extensions Tim: I don't get the feeling that anyone on the call strongly feels we should add support for queries to begin with @ Greg: I agree so long as we can update this spec at some later point Stefan: I think we should only support $ for now **Consensus**: Queries should not begin with @, but this is a natural extension point. **Action**: Chairs to update the Github issue and allow for non-attendees to reply **Action**: Write an extensibility model ## Regular Expressions in Filters - #70 Greg: There is two aspects here, do we support them and what kind do we support? Carsten: We should answer the second aspect first Carsten: Two kinds of regular expressions - matching vs/ computing And that we need a mechanism for matching Carsten: Need to refer to existing standard and subset Tim: Is there anyone who wants regular expressions removed Marko: That would be a deal breaker, but it would be hard to make something interoperable Greg: As an implementor my preference would be get a library that does regular expressions for me, matching exactly what we specify Carsten: For a library developer adding a feature is a win, for a standards developer is a lose, lose, lose Greg: This is a big enough feature worth including, we just have to figure out how Tim: Unless we can have a specification of what we mean, we shouldn't have regular expressions Those who want regular expressions need to propose exactly what that means Glyn: If we pick a subset of W3C, any implementation would have to massage that into implementations Tim: Are there any other IETF RFCs that use regular expressions? Carsten: YANG for example defines using W3C, but implementations use PCRE Marko: When we say subsets, does the standard define invalid/malformed expression? Would the standard require the library to refuse it? Carsten: That is answered by the extensibility concept Tim (summary): If we get a specific proposal, we can go ahead Stefan: Computed REs or literal REs only, proposal to stick with literal only now Carsten: It's likely an extension would come up Glyn: DoS issues? Could make our subset a common subset of W3C and RE2 to address that. **Consensus**: Regular expressions can be included once we have a specific proposal **Consensus**: Only permit literals in regular expressions **Action**: Carsten to attempt a PR **Action**: Cover DoS in the security considerations of the document ## Duplicates in Selector output - #23 Greg: Two of the same value in the input, if it's extensive object it doesn't make sense remain portion of the path Questions raised around identifying not only value but location If somehow the path evaluated the same location in both, those would be removed ... but if the same value but different location, those would remain in the output Carsten: Can you clarify if by "same" you mean "equal" or "identical"? Greg: Two nodes with the same value should not be collapsed, but two selectors return the same node, those should be collapsed What I wrote on the issue is not what we landed on, what I wrote was strict value equality not node equality. From a performance perspective I don't want to evaluate the same object twice Stefan: I agree with this and see overall agreement Tim: When I'm writing a JSONPath usually I know the JSON I'm writing against looks like I'm having trouble imaging a scenario where I would want duplicate removal to happen Greg: The argument of identifying a count of something is a really good argument against this Tim: The removal of duplicates in something that happens at another level of the program Greg: Maybe this is out of scope, and let the application handle this afterwards Glyn: I would be very happy to allow duplicates Carsten: We may have a simplified view of expressions actually mean, and might assume duplicates being removed Stefan: I can agree to just allow duplicates for now Marko: Isn't descendants selector kind of implicitly removing duplicates? Greg: It wouldn't remove them Tim: The draft does not well describe this **Consensus**: Duplicate removal won't be part of the document **Action**: Marko to write the descendants PR ## Filter Expressions - #64 Carsten: terminology: filter selectors contain filter expressions (vs. indexing selectors that contain indexing expressions, which may or may not be similar) Stefan: I proposing first we do the filter expressions, then return to indexing expressions I'm not sure if it's possible to do valid parsing without the parenthesis Greg: Lots of questions in the air, we should coalesce the related into a single issue #17, #57, #92 It might help for someone to condense these down into list of open questions Carsten: These should be several issues in the end Stefan: I don't see the necessity in doing them in new issues Greg: Maybe we can look at closing the other ones, there's a lot to unfold Stefan: One central discussion point is the filter selector expression syntax, which must resolve to a boolean expression I would start with just using comparison and boolean operators Greg: However the current node is equal to some other node that I reference from the root minus 1, I should be able to do that Tim: We should look at support in existing implementations Greg: If you're going to be passing boolean operations it's not a big stretch to support these arithmetic operations Glyn: Numeric operations don't pay for the cost, agree implementation wouldn't be too bad but increased verbiage in the specification Tim: If it turns out this is not widely supported, I'd be negative of us inventing it Stefan: Also comparing weak or strong type comparisons, I would prefer weak type Greg: I imagine loose typing to be divisive, and would prefer strict equality Glyn: Does anyone support loose equality? Marko: What we have to define is what happens when input has incompatible types, and how to model the behaviour of ignoring those cases Greg: We already have verbiage in the specification to that effect Stefan: Is it our task to do provisions to avoid security issue point, or the task of the implementors Greg: Either the implementors or the client of the implementation It's our responsibility to highlight known potential risks I don't know if it our responsibility to mitigate those risks Carsten: If avoiding the risk is feasible, we should avoid it Glyn: Another consideration is non-scalar operations Tim: I would like to rule it out, as it's a fruitful source of baffling behaviour Marko: What about lexicographical sorting or collation? (for > etc.) Tim: Unicode code points, or madness Marko: We have to say that explicitly Stefan: Can we agree to use for operators C-based syntax Tim: Do check existing implementations **Action**: Stefan to write PR # AOB ## Next Interim meeting **Action**: Chairs to arrange a Doodle for an interim meeting in mid-June