Manually Creating Your AutoPOST URL

It's pretty easy if you understand how form variables are encoded in a URL, though still not as easy as using the AutoPOST URL generator. I'm including this page for all you obsessive hackers, or if for some reason the AutoPOST URL generator doesn't work on your page (in which case, let me know).

For forms that use GET:

It's always been possible to automate GET submission; AutoPOST just makes it easier by generating the URL for you. The extended URL is already defined in RFC 1738, and the CGI definition. The anatomy of this URL is

Target-URL?GET-data

where Target-URL is the script that the form calls, and GET-data is the form data to be submitted, in the form "name1=value1&name2=value2&name3=value3". For more info on the GET-data, and how to encode it, see the next section-- it takes exactly the same form as POST-data.

For forms that use POST:

To automate POST submissions, you gotta call the main AutoPOST program each time you submit the form. The program receives the data and forwards it to your target URL as a POST submission.

The anatomy of the AutoPOST extended URL is "AutoPOST-script-URL/Target-URL?POST-data", or specifically,

http://www.io.com/~jsm/nph-ap.cgi/Target-URL?POST-data

Think of it as three fields, separated by a slash and a question mark. These fields are:

The URL of the AutoPOST program, http://www.io.com/~jsm/nph-ap.cgi;
The URL you are posting to (generally a script); and
The data you want to POST to that URL, obtained by examining the form in the HTML file. Note that the data must be correctly URL-encoded (see the next section if this is unclear).

(Notice that the URL of the input form is not needed at all in the final AutoPOST URL.)

For example, to automatically query WebCrawler for "sex", "all keywords", and 25 hits, use the URL

http://www.io.com/~jsm/nph-ap.cgi/http://webcrawler.com/cgi-bin/WebQuery?searchText=sex&andOr=all&maxHits=25

Yes, that makes a long URL. Hopefully, it won't be too long for your browser-- the size limit for GET data is much smaller than for POSTed data, but still enough to accommodate most form submissions. (This is a fundamental limit of AutoPOST, because of the GET mechanism. I've tried to make the AutoPOST URL as short as possible, to maximize the number of characters you can use for your own URL and POST data. There are tricks to eek out a little more space-- ask me if you need to know how.)

(The other limit of AutoPOST is that it won't support sites that require the Netscape-specific cookie feature. Cookies are an HTTP extension used to personalize your interaction with a web site.)

Whoa, hold on! What's this about URL-encoding? How did you figure out what POST data to send? And where did the & and = signs come from?

OK, OK. I admit, you need to be somewhat HTML-fluent, especially regarding forms, to manually create the URL. I won't give you a complete lesson, but here are some details to get you started:

First, load up the Web page with the form you want to automate the POST from. Then, View the Document Source (an option in most browsers).
The URL to post to (item 2, above) can be gotten from the <FORM...> tag, after the "ACTION=" attribute.
The data to post (item 3, above) can be found in all the <INPUT...>, <TEXTAREA...>, and <SELECT...> tags between the <FORM...> and the </FORM> tags. The data can then be assembled into one long string as follows:
- Each <INPUT...>, <TEXTAREA...>, or <SELECT...> tag or set of tags defines one data field, which is given an identifying name by the "NAME=" attribute.
- Each field has a value, which is entered or selected by the user.
- String these name-value pairs together to make up the POST data, in the form "name1=value1&name2=value2&name3=value3", etc.
- To finish the URL-encoding of the data, perform the following two steps:
  1. Change all " " (spaces) in the string to + (plus signs)
  2. If you have any funky characters (defined below) in the names or values, replace them with %xx, where xx is the ASCII code of that funky character, in hex. Do NOT replace the & and = signs that separate the name-value pairs.
    Funky characters are &, =, +, %, and ?. Replace them with %26, %3D, %2B, %25, and %3F respectively. If any other characters might give you trouble, escape them in the same way-- there is no danger from escaping too many characters.
You now have the POST-data you need for your AutoPOST URL, to put after the final question mark.

This whole process may get a little messy, to be sure. But you only have to do it once per bookmark, and it can be made easier with jucidious cutting and pasting. And, I'm open to suggestions on how to improve it. (Really, though, you should be using the automatic URL generator, if you're having trouble with this manual stuff.)

For help with HTML forms, try this page at UIUC. For help with URL encoding, try the same document, in the section called "Form Submission". For the full definition of URL encoding, see RFC 1738, Section 2.2.

Go back to main AutoPOST page