Repository navigation
Conversation
…clarations BaseParser::NAME used \w, which is ASCII-only in Ruby, so <!DOCTYPE été>, <?été ...?>, <!ENTITY été ...>, &été;, <!NOTATION été ...> and non-ASCII ATTLIST names raised ParseException, although element and attribute names already accept non-ASCII letters. Use the XML Name production from XMLTokens (NameStartChar and NameChar), which REXML::Entity and REXML::Text already use. Names starting with a digit, which [5] Name does not allow, are now rejected in these places too.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BaseParser::NAMEis built from\w, which is ASCII-only in Ruby regular expressions. So every name parsed with it rejects non-ASCII letters, while element and attribute names (QNAME, built from[:alpha:]) accept them:xml.etree)<!DOCTYPE été><r/>Malformed DOCTYPE: invalid name<?été content?><r/>Invalid processing instruction node: invalid name<!DOCTYPE r [<!ENTITY été "summer">]><r>&été;</r>Malformed entity declaration"summer""summer"<!NOTATION été PUBLIC 'urn:x'>in the internal subsetMalformed notation declaration: invalid name<!ATTLIST a été CDATA #IMPLIED>in the internal subsetBad ATTLIST declaration!This change defines
NAMECHARandNAMEwithXMLTokens::NAME_CHARandXMLTokens::NAME, the [4]/[4a]/[5] productions thatREXML::EntityandREXML::Textalready use. The capture group is the same, so the patterns that interpolateNAMEkeep their group numbers. One side effect: names that start with a digit, which [5] Name doesn't allow, are now rejected in these places too, as element names already are.I found this with the W3C XML Conformance Test Suite (xmlts20130923). The three
japanese/valid documents (UTF-8 and UTF-16 with Japanese names in the DTD) now parse, and two not-wf tests (oasis p58fail3, IBM P66 ibm66n03) are now rejected.Tests:
test_non_asciifor DOCTYPE names and general entity names,test_non_ascii_namefor notations, andtest_non_ascii_targetfor processing instructions. All four fail before the change and pass after it.bundle exec rake testpasses (845 tests), as does the--enable-frozen-string-literalrun.ruby -wshows no new warnings.Performance: a 5000-element document doesn't use these patterns for tags, so its parse time is unchanged. For a document with 500 entity, ATTLIST and NOTATION declarations, 2000 PIs and entity references, the difference was within run-to-run noise. A name scan in isolation is about 20% slower (about 0.25 µs per scan).