Introduction
    hi all
    This is a small function for handling POST and GET queries natively in AHK so external utilities to handle it are not needed any more. It can be used where no additional utilities, such as cURL or wget are wanted.
    The latest version handles following features:

    • ports in URL
    • Username:Password@domain style URLs (see this answer to avoid common problems)
    • SSL aka https
    • HeaderInformations / Dumping / Storing
    • DownloadSizes / CurrentSize (for download progressbars)
    • Flags for InternetConnection handling (auto follow feature etc)
    • Referrer for Queries
    • Accepttypes for Queries
    • Proxy handling
    • timeout handling
    • custom UserAgent


Usage

    Usage is pretty simple:
    You need to define a Variable which will receive the returned databuffer.
    When only Strings are expected as returnvalues, autohotkey needs to learn the new size, so a VarSetCapacity(buffer,-1) is needed. This has been modified to support binary downloads aswell.
    httpQUERY(buffer:="",URL) will result the length of the standard HTML which is usually retrieved. The html itself is stored in the variable buffer. The query method is GET as the length of Postparams is zero.
    httpQUERY(buffer:="",URL,POSTDATA) will work as the GET above just with POSTDATA being transmitted. The query method is POST as the length of Postparams is non zero.
    The URL now supports following scheme:
    http://username:password@server.tld:80/path/to/target.ext?queryops#fragment

    since httpQuery has been updated to use InternetCrackURL from winINet, all essential parts will be recognized. so there is no need to set up any additional parameters. Attention: When u need to authetificate in the Website the username / password attempt will not work. u have to submit those parameters via POST or GET method.

    Additional Parameters:
    To see a dump of the received httpHeaders, there is buildIn support for a global Variable named httpQueryOps. It may consist of one or more Verbs. For now "showHeader", "storeHeader", and "updateSize" verbs are supported. If You use storeHeader the complete Header will be saved in a variable named HttpQueryHeader which will be made global at runtime. The verb updateSize will make two variables globally Available: httpQueryFullSize and httpQueryCurrentSize. An usage example to show a download status indicator is included

    Following Variables are evaluated globally:
    httpAgent: can hold an individual UserAgent Code. The default is AutoHotkeyScript
    httpProxy: the name of a proxy server to use. default = 0
    httpProxyByPass: list of domains which will not be used with the proxy. default = 0
    httpQueryReferer: an URL which is recognized by QueriedServer, as the location where the Request was generated mind though its "Referrer" the name of the variable is httpQueryReferer [sic]
    httpQueryAcceptType: this variable lets u specify your accepted stream formats for the results of the query
    httpQueryDwFlags: if in need for any special flags for the current connection this is the variable to set (example V shows an useCase for this feat)

    I think these are pretty self explaining.

Examples

    Example I: Showing the raw HTML result of a search for httpQuery in AutoHotkey's forum in an edit control

Code (Expand):
; exmpl.searchAHKforum.httpQuery.ahk
; Searches the forum for a given Phrase: in this case httpQuery
#noenv
html     := ""
URL      := "http://www.autohotkey.com/forum/search.php?mode=results"
POSTData := "search_keywords=httpQuery&search_terms=all&search_forum=-1&"
          . "search_time=0&search_fields=all&show_results=topics&return_chars=500"
          . "&sort_by=0&sort_dir=DESC"

length := httpQuery(html,URL,POSTdata)
varSetCapacity(html,-1)

Gui,Add,Edit,w600 +Wrap r25,% html
Gui,Show
Return

GuiClose:
GuiEscape:
   ExitApp
   
#include httpQuery.ahk

    Remarks: To make a function work with POST parameters, it is essential to know which parameters are actually sent. These can be obtained by studying the HTML source of the target URL or by using analyzing tools such as the HttpFox Addon for Firefox. You might also want to use the following example to learn the nature of the form in a HTML document


    Example II: Showing a dump of a HTML form to help making valid POST parameters

Code (Expand):
; exmpl.formdump.httpquery.ahk
; Form Dumper v0.1b (w) 9th July 2008 by derRaphael
#NoEnv
InputBox
 ,URL                                                                  ; OutputVariable
 ,Enter URL to analyze                                                 ; Title of box
 ,Please enter the complete URL starting with http:// to be analysed   ; Descriptive text
 ,,,,,,,,http://www.autohotkey.com/forum/search.php                    ; default value

   html := ""
   htmlLength := httpQuery(html,URL)
   VarSetCapacity(html,-1)

; The Complete Form Node from given URL's HTML
   RegExMatch(html,"i)<form.+?</form>",formNode)
; Just the formtag for further analyzing
   RegExMatch(formNode,"i)<form[^>]+?>",formTag)
; The name of the form
   RegExMatch(formTag,"i)NAME=""?(?P<Name>.+?)""?>",form)
; The method used to process Data
   RegExMatch(formTag,"i)METHOD=""?(?P<Method>.+?)""?\s",form)
; The complete address used to send data to
   RegExMatch(formTag,"i)ACTION=""?(?P<Action>.+?)""?\s",form)
; just the url
   RegExMatch(formAction,"i)(?P<URL>[^\?]+)",formA_)
; any existing GET parameters
   RegExMatch(formAction,"i)\?(?P<GET>.*)",formA_)
   ; Fix &amp; to & as delimiter
   StringReplace,formA_GET,formA_GET,&amp`;,&,All

   startPosI := startPosS := startPosT := 0
   inpCount := selCount := txtCount := 0

   Loop,
   {
      If (startPosI:=RegExMatch(formNode,"i)<input[^>]+>",inputTag,startPosI+1)) {
         inpCount++
         formInput%InpCount% := inputTag
      } else {
         noMoreInput := 1
      }
      If (startPosS:=RegExMatch(formNode,"i)<select.*?</select>",selectNode,startPosS+1)) {
         selCount++
         formSelect%selCount% := selectNode
      } else {
         noMoreSelectOptions := 1
      }
      If (startPosT:=RegExMatch(formNode,"i)<textarea.*?</textarea>",textareaNode,startPosT+1)) {
         txtCount++
         formTextarea%txtCount% := textareaNode
      } else {
         noMoreTextArea := 1
      }
      if (NoMoreInput) && (NoMoreTextarea) && (NoMoreSelectOptions) {
         break
      }
   }
   Loop,% inpCount
      Inputs .= formInput%A_Index% "`n"
   StringReplace,inputs,inputs,<,%A_Tab%<,All
   Loop,% selCount
      Selects .= formSelect%A_Index% "`n"
   StringReplace,Selects,Selects,</option>,</option>`n,All
   StringReplace,Selects,Selects,<option,%A_Tab%<option,All
   Selects := RegExReplace(selects,"i)<select[^>]+>","$0`n")
      
   Loop,% txtCount
      txtAreas .= formTextArea%A_Index% "`n"
   StringReplace,txtAreas,txtAreas,</textarea>,</textarea>`n,All
   StringReplace,txtAreas,txtAreas,<textarea,%A_Tab%<textarea,All

; dump results to Gui
Gui,Add,Tab2, w800 h20,Analyzed Content|found Form|raw HTML
Gui,Tab,1
Gui,Add,Edit,w800 h600 yp+20 xp , % ""
      . "formTag:`n" formTag "`n`n"
        . "formMethod:`t" formMethod "`n"
        . "formAction:`t" formAction "`n"
        . "formActionURL:`t" formA_URL "`n"
        . "formActionGET:`t" formA_GET "`n`n"
      . "Total Inputs:`t" inpCount "`n"
      . inputs
      . "`nTotal Selects:`t" selCount "`n"
      . selects "`n"
      . "`nTotal TextAreas:`t" txtCount "`n"
      . txtAreas "`n"
Gui,Tab,2
Gui,Add,Edit,w800 h600 yp xp , % formNode
Gui,Tab,3
Gui,Add,Edit,w800 h600 yp xp , % html
Gui,Show,,Statistics for Form: "%formName%" (%URL%)
return

GuiClose:
GuiEscape:
   ExitApp
   
#include httpQuery.ahk

    Remarks: Using this function provides informations of which elements are used in a form which will be submitted. It neither catches modded JavaScript content nor more than one form. It has been tested with google.com, autohotkey.com's searchform and autohotkey.net's pastebin


    Example III: Downloading binary data and saving it to a local file.

Code (Expand):
; exmpl.downloadBinary.httpQuery.ahk
; This example downloads the latest AHK environment and stores
; the received binary data to a file.
#noenv
data     := ""
URL      := "http://www.autohotkey.net/programs/AutoHotkey104706.zip"
httpQueryOps := "updateSize"
SetTimer,showSize,10
length   := httpQuery(data,URL)
Tooltip
if (write_bin(data,"ahk.exe",length)!=1)
   MsgBox "There was an Error!"
else
   MsgBox AHK Source downloaded and saved as "ahk.zip"!
Return

showSize:
   Tooltip,% HttpQueryCurrentSize "/" HttpQueryFullSize
return

GuiClose:
GuiEscape:
   ExitApp

write_bin(byref bin,filename,size){
   h := DllCall("CreateFile","str",filename,"Uint",0x40000000
            ,"Uint",0,"UInt",0,"UInt",4,"Uint",0,"UInt",0)
   IfEqual h,-1, SetEnv, ErrorLevel, -1
   IfNotEqual ErrorLevel,0,ExitApp ; couldn't create the file
   r := DllCall("SetFilePointerEx","Uint",h,"Int64",0,"UInt *",p,"Int",0)
   IfEqual r,0, SetEnv, ErrorLevel, -3
   IfNotEqual ErrorLevel,0, {
      t = %ErrorLevel%              ; save ErrorLevel to be returned
      DllCall("CloseHandle", "Uint", h)
      ErrorLevel = %t%              ; return seek error
   }
   result := DllCall("WriteFile","UInt",h,"Str",bin,"UInt"
               ,size,"UInt *",Written,"UInt",0)
   h := DllCall("CloseHandle", "Uint", h)
   return, 1
}

#include httpQuery.ahk

    Remarks: This example downloads the current Autohotkey Source and saves it to local disk as ahk.zip in the same path as the script. While it does so, it shows the current progress of the download



    Example IV: Uploading an image to Imageshack using the official (free) API

Code (Expand):
; exmpl.imageshack.httpQuery.ahk
; This example uploads an image and constructs a multipart/form-data Type
; for fileuploading and returns the XML which is returned to show the stored Imagepath
   FileSelectFile,image
   FileGetSize,size,%image%
   SplitPath,image,OFN
   FileRead,img,%image%
   VarSetCapacity(placeholder,size,32)
   boundary := makeProperBoundary()
   post:="--" boundary "`ncontent-disposition: form-data; name=""MAX_FILE_SIZE""`n`n"
      . "1048576`n--" boundary "`ncontent-disposition: form-data; name=""xml""`n`nyes`n--"
      . boundary "`ncontent-disposition: form-data; name=""fileupload""; filename="""
      . ofn """`nContent-type: " MimeType(img) "`nContent-Transfer-Encoding: binary`n`n"
      . placeholder "`n--" boundary "--"
   headers:="Content-type: multipart/form-data, boundary=" boundary "`nContent-Length: " strlen(post)
   DllCall("RtlMoveMemory","uInt",(offset:=&post+strlen(post)-strlen(Boundary)-size-5)
         ,"uInt",&img,"uInt",size)
   size := httpQuery(result:="","http://www.imageshack.us/index.php",post,headers)
   VarSetCapacity(result,-1)
   Gui,Add,Edit,w800 h600, % result
   Gui,Show
return

GuiClose:
GuiEscape:
   ExitApp

makeProperBoundary(){
   Loop,26
      n .= chr(64+a_index)
   n .= "0123456789"
   Loop,% StrLen(A_Now) {
      Random,rnd,1,% StrLen(n)
      Random,UL,0,1
      b .= RegExReplace(SubStr(n,rnd,1),".$","$" (round(UL)? "U":"L") "0")
   }
   Return b
}

MimeType(ByRef Binary) {
   MimeTypes:="424d image/bmp|4749463 image/gif|ffd8ffe image/jpeg|89504e4 image/png|4657530"
          . " application/x-shockwave-flash|49492a0 image/tiff"
   @:="0123456789abcdef"
   Loop,8
      hex .= substr(@,(*(a:=&Binary-1+a_index)>>4)+1,1) substr(@,((*a)&15)+1,1)
   Loop,Parse,MimeTypes,|
      if ((substr(hex,1,strlen(n:=RegExReplace(A_Loopfield,"\s.*"))))=n)
         Mime := RegExReplace(A_LoopField,".*?\s")
   Return (Mime!="") ? Mime : "application/octet-stream"
}

#include httpQuery.ahk

    Remarks: Uploading binary data where an encodingtype "multipart/form-data" was needed, didnt work with classic POST usage. this example shows how to build the complete POST request in a different manner and to include raw binary data with no encoding such as base64 or ascii85



    Example V: Modifying dwFlags for special useCases where a 302 is the returncode such as with dynamic created content urls

Code (Expand):
#NoEnv
   Gui,add,edit,w800 h400 vPasteBin, this is a testtext!
   Gui,Add,Button,gPaste wp, Upload this text to http://pastebin.com
   Gui,Show
return

Paste:
   Gui,Submit,Nohide
   URL := "http://pastebin.com/pastebin.php"
   POSTDATA := "parent_pid=&format=text&code2=" uriEncode(PasteBin) "&"
           . "poster=&paste=Send&expiry=d&email=&"
   httpQueryOps := "showHeader storeHeader"
   httpQueryDwFlags := (INTERNET_FLAG_NO_AUTO_REDIRECT:= 0x00200000)
   length := httpQuery(HTML:="",URL,POSTDATA)
   VarSetCapacity(HTML,-1)
   gui,destroy
   Gui,Add,Edit,w800 h600,% HttpQueryHeader "`n`n" html
   Gui,Show
Return

GuiClose:
GuiEscape:
   ExitApp

uriEncode(str)
{ ; v 0.3 / (w) 24.06.2008 by derRaphael / zLib-Style release
   b_Format := A_FormatInteger
   data := ""
   SetFormat,Integer,H
   Loop,Parse,str
      if ((Asc(A_LoopField)>0x7f) || (Asc(A_LoopField)<0x30) || (asc(A_LoopField)=0x3d))
         data .= "%" . ((StrLen(c:=SubStr(ASC(A_LoopField),3))<2) ? "0" . c : c)
      Else
         data .= A_LoopField
   SetFormat,Integer,%b_format%
   return data
}

#include httpQUERY.ahk

    Remarks: Some WebServices such as PasteBin.com generate Dynamic Location Urls in order to show processed content. Using httpQuery in a default manner, results in a 404 Not found error when the script tries to access the location to fast. Specifying the NO_FOLLOW dwFlag solves this problem but needs a 2nd request to grab the fresh received Location Header.