The HTTP Query Method
ietf.org259 points by Ivoah 7 days ago
259 points by Ivoah 7 days ago
PSA: When posting an RFC or (especially) an RFC-draft, please use the IETF Datatracker URL.
For example, this one is: https://datatracker.ietf.org/doc/draft-ietf-httpbis-safe-met...
The best way to view an RFC, IMHO, is to use the "htmlized" format: you can view and compare different versions, view errata for a formal RFC, and go back to Datatracker at any time.
Also, the Datatracker URL is version-insensitive, so unlike the pure HTML format, it will not be stuck on draft-14 forever.
> please use the IETF Datatracker URL.
On my phone, your Datatracker link results in an unreadable mess of a page due to the hard-coded line breaks in the plaintext rendition of the RFC text (making it unreadable in portrait mode) and the huge sticky page nav (causing the content viewport to shrink vertically to almost zero in landscape mode). The HTML page behind OP's link reads just fine.
> The best way to view an RFC, IMHO, is to use the "htmlized" format
I don't see any choices of format such as HTML behind your link. There's a sticky nav, then a couple of pages of metadata, followed by a plaintext rendering of the RFC. What am I missing?
When I open the link there is a table of information before the main RFC text. One row is titled ‘formats’ in bold and has a ‘htmlized’ link in it. Presumably that is what you were missing due to some understandable banner blindness.
In the top section where there’s authors etc, there’s a format list. I didn’t see it the first time.
It reads decently in landscape mode, and i’m on a small screen (iPhone se 3rd gen).
It shows me only seven lines of text at a time [0], too few for me to read comfortably, let alone make sense of e.g. diagrams. Firefox on Librem 5.
To quickly retrieve an RFC I use a script named "rfc". I save RFCs as text files not HTML so I prefer the .txt version
echo https://www.ietf.org/rfc/rfc$1.txt|yy025|nc -vv 6f6b 80|yy045|less
6f6b is TLS forward proxy
yy045 removes chunked encoding
less is more(1)
For drafts I use a metasearch script
+1 for non-paginated results
;;64) #``````````` datatracker
shift
x=$(echo "$x"|sed 's/ /+/g;s/++/+/g')
echo "https://datatracker.ietf.org/doc/search?name=$x&activedrafts=on&olddrafts=on"|yy025|(yy084 64 "$x";nc -vv 6f6b 80)
exit
From the Datatracker, I can see that it was called SEARCH until draft-2 (Nov 2021), and then changed to QUERY.
Also, the previous SEARCH method was proposed in Apr 2015 (!!), but nobody took it seriously, and it never gained traction back then. A lot of software was developed/updated during the last decade, and a lot of opportunities were missed. Even if the QUERY method is turned into a formal RFC right now, expect 10+ years for everyone to adopt it, without running into HTTP-405 or 501's.
It just takes some significant piece of server software to start accepting it, RFC or not. "Write code, not laws", or however was that formulated.
See the history of the PATCH method, and of the whole WebDAV thing.
I can’t wait for QUERY to become an official RFC.
It's felt quite awkward to tiptoe around the existing spec when building features that retrieve data; we've had to either use POST to keep sensitive filter criteria out of http logs or just create a usually massive URL-encoded query string.
There's nothing holding you back implementing the QUERY method right now - HTTP methods are standardized, but not limited to the standard. Obsviously it depends how proxies/servers/etc. might handle uncommon methods.
But that's the point, isn't it? Browsers, proxies and servers always assume POST isn't idempotent. When the user presses F5 the browser asks if they want to do the thing again. You can't prevent that without making it more complicated (e.g. send the request from JavaScript).
> There's nothing holding you back implementing the QUERY method right now - HTTP methods are standardized, but not limited to the standard.
I think this comment is terribly naive. Technically you can write your own service to support verbs like LOL or FUBAR, but in practice your service is not only expected to be broken when passing requests through other participants you do not control but also it requires far more development work to integrate with existing standards. Take for example CORS. If a HTTP method is not deemed safe then client requests need to go through the unsafe flow with preflight requests and the like. Forget support for caching too, and good luck having your requests pass through proxies.
So what exactly did you achieved by using non-standard verbs?
If you chose to go the ignorant backwards incompatible way, you are better off not using HTTP at all and just go with some random messaging/RPC protocol.
I won't be so strict. Even though a homebrew implementation won't be widely interoperable, an experience of its active development and use in a limited environment (e.g. within a company) would be valuable both to inform the RFC and to serve an example implementation.
Very timely as I just recently ended up with a URL query string so big that CloudFront rejected the request before it even hit my server.. Ended up switching that endpoint to POST. Would've liked QUERY for that!
I have come across systems that use GET but with a payload like POST.
This allows the GET to bypass the 4k URL limit.
It's not a common pattern, and QUERY is a nice way to differentiate it (and, I suspect will be more compatible with Middleware).
I have a suspicion that quite a few servers support this pattern (as does my own) but not many programmers are aware of it, so it's very infrequently used.
Sending a GET request with a body is just asking for all sorts of weird caching and processing issues.
I get the GPs suggestion is non-conventional but I don’t see why it would cause caching issues.
If you’re sending over TLS (and there’s little reason why you shouldn’t these day) then you can limit these caching issues to the user agent and infra you host.
Caching is also generally managed via HTTP headers, and you also have control over them.
Processing might be a bigger issue, but again, it’s just any hosting infrastructure you need to be concerned about and you have ownership over those.
I’d imagine using this hack would make debugging harder. Likewise for using any off-the-shelf frameworks that expect things to confirm to a Swagger/OpenAPI definition.
Supplementing query strings with HTTP headers might be a more reliable interim hack. But there’s definitely not a perfect solution here.
To be clear, it's less of a "suggestion" and more of a report of something I've come across in the wild.
And as much as it may disregard the RFC, that's not a convincing argument for the customer who is looking to interact with a specific server that requires it.
Cache in web middleware like Apache or nginx by default ignores GET request body, which may lead to bugs and security vulnerabilities.
But as I said, you control that infra.
I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved.
If you can’t trust them to do that little, then you’re fuck regardless of whether you decide to send payloads as GET bodies.
And there isn’t any good reason not to contract pen testers to check over everything afterwards.
> I don’t think it’s unreasonable to expect your sysadmins, devops, platform engineers, or whatever title you choose to give them, to set up these services correctly, given it’s their job to do so and there’s a plethora of security risks involved.
Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist: "content received in a GET request has no generally defined semantics, cannot alter the meaning or target of the request, and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack" (RFC 9110)
> And there isn’t any good reason not to contract pen testers to check over everything afterwards.
I am pretty sure our SecOps and Infra Ops and code standards committee will check it and declare that GET bodies is a hard no.
> Exactly, and the correct way to setup GET requests is to ignore their bodies for caching purposes because they aren't expected to exist
No. The correct way to set up this infra is the way that works for a particular problem while still being secure.
If you’re so inflexible as an engineer that you cannot set up caching correctly for a specific edge case because it breaks you’re preferred assumptions, then you’re not a very good engineer.
> and might lead some implementations to reject the request and close the connection because of its potential as a request smuggling attack"
Once again, you have control over the implementations you use in your infra.
Also It’s not a RSA if the request is supposed to contain a payload in the body.
> I am pretty sure our SecOps and Infra Ops and code standards committee will check it and declare that GET bodies is a hard no.
I wouldn’t be so sure. I’ve worked with a plethora of different infosec folk from those who will mandate that PostgreSQL needs to use non-standard ports because of mandating strict compliance with NIST, even for low risk reports. To others that have been fine with some pretty massive deviations from traditionally recommended best practices.
The good infosec guys, and good platform engineers too, don’t look at things in black and white like you are. They build up a risk assessment and judge each deviation on its own merit. Thus GET body payloads might make sense in some specific scenarios.
This doesn’t mean that everyone should do it nor that it’s a good idea outside of those niche circumstances. But it does mean that you shouldn’t hold on to these rigid rules like gospel truths. Sometimes the most pragmatic solution is the unconditional one.
That all said, I can’t think of any specific circumstance where you’d want to do this kind of hack. But that doesn’t mean that reasonable circumstances would never exist.