Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client certificate support in clj-apache-http #7

Open
wants to merge 51 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
9772565
Initial checkin. Sample code is known to work.
May 10, 2011
b7ad711
Fixed some formatting in the README
May 10, 2011
ab30747
Typo
May 10, 2011
014f566
More README fixes.
May 10, 2011
7df910a
Made some updates that allow this package to be published via Clojars.
May 11, 2011
34d115c
Throw a meaningful exception when we can't load a file or resource.
May 11, 2011
0cfd36c
Added make-clojar.sh. Removed comments in project.clj. No substantive…
Jun 8, 2011
d1a84d0
Starting work on async client. Currently very unstable!
Jun 13, 2011
31c481b
Interim checkin. Callbacks are working.
Jun 14, 2011
e705ffa
Checkpoint. Refactoring.
Jun 14, 2011
e1108f3
Checkpoint. Added some tests for async client.
Jun 14, 2011
f543395
Updated files required to build.
Jun 14, 2011
48107ec
Updated description to mention alpha async client support.
Jun 14, 2011
f269959
Working on async features. The async API is still unstable, so beware!
Jun 15, 2011
cb9dda4
Fixed a few problems with the async test. Still a work in progress.
Jun 15, 2011
df24185
Added more flexible API for running async.
cvillecsteele Jun 15, 2011
cef0e3a
Merge pull request #1 from g1nn13/async
Jun 15, 2011
8d00774
Fixed create-http-params so it sets the client parameters that the ca…
Jun 15, 2011
6d0f5d5
Merge branch 'async' of github.com:diamondap/clj-apache-http into async
Jun 15, 2011
2976aa1
Set version to 1.0.4
Jun 15, 2011
17de019
Refactored run! to make more sense in the context of running multiple
cvillecsteele Jun 16, 2011
f214693
Merge pull request #2 from g1nn13/async
cvillecsteele Jun 16, 2011
cb67125
Init context for ssl correctly.
cvillecsteele Jun 16, 2011
e3f48bf
Changed init-scheme-registry to create-scheme-registry.
Jun 16, 2011
ed1fb42
Changed name of param from client-options to http-params
Jun 16, 2011
27af7ef
Version 1.0.5
Jun 16, 2011
85ae42e
Added http-params param to http-client
Jun 16, 2011
3d79b74
Setting http-params on the http-connection now seems to be working pr…
Jun 16, 2011
9c4e1dd
Refactoring to allow x509 certificate manager to work with both sync …
Jun 20, 2011
a1db3aa
Version 1.10 supports sync and async clients with x509 certs.
Jun 20, 2011
64b1ebc
Fixed type hint on create-http-params.
Jun 24, 2011
61209cb
Added some documentation on the async client. Reorganized existing doc.
Jun 28, 2011
292a9de
Fixed some doc formatting.
Jun 28, 2011
ab10cbd
New build merges async branch into master. Fixes an issue with the way
Jun 28, 2011
6c94b7f
Fixed routes per host setting. Set version to 1.0.12
Jun 30, 2011
013ef9b
Added internal exception handler for IOReactor.
Jul 5, 2011
b7177ee
Changed version to 1.0.13
Jul 5, 2011
6d2767e
Removed CountdownLatch from run! (though execute-batch still uses it)…
Jul 6, 2011
9d15c62
Added param for keep-alive. Wrapped test code in comment.
Jul 8, 2011
bda931d
Added doc on timeouts.
Jul 8, 2011
ca41df6
Changed version to 1.0.14
Jul 8, 2011
07731ca
Added import IOSession
Jul 11, 2011
fd5a18a
Fixed type hints to enable loading resources from jars
cvillecsteele Sep 21, 2011
7a5f3f6
Getting repo details straight.
cvillecsteele Sep 21, 2011
592fbb2
Remove dependency on clojure-contrib to make this friendlier for Cloj…
Oct 10, 2011
ddb1618
Merge pull request #3 from revelytix/master
Oct 11, 2011
abf438e
Bumped version to 1.0.16.
Oct 11, 2011
e41c92f
Added MIT license.
Oct 11, 2011
2b72740
Changed version of Apache async client lib from 4.0-alpha2 to 4.0-alpha3
Nov 3, 2011
28d57c9
Rolled back last set of changes. 1.0.18 is now the same as 1.0.17
Nov 3, 2011
3da8d66
Update README.markdown
diamondap Sep 14, 2012
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions ChangeLog.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
== version 1.0.18
* moved back to verison 4.0-alpha2 of org.apache.httpcomponents/httpasyncclient - this makes 1.0.18 identical to 1.0.17

== version 1.0.17
* moved from version 4.0-alpha2 to version 4.0-alpha3 of org.apache.httpcomponents/httpasyncclient
19 changes: 19 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Copyright (C) 2011 by hotelicopter.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
218 changes: 23 additions & 195 deletions README.markdown
Original file line number Diff line number Diff line change
@@ -1,208 +1,36 @@
# What is it? #
# *** DEPRECATED *** #

`clj-apache-http` is a Clojure wrapper library for the Apache HTTP Client
(version 4.0).
This library is built on an alpha version of the Apache HTTP async client.
That underlying alpha library has some race conditions that cause connections
to leak from the connection pool. Under heavy use, you may lose all connections
in the pool.

<http://hc.apache.org/>
The leaks were fixed in later versions of the underlying Apache library,
but those versions are not compatible with the alpha upon which this library was
built.

It defines functions to perform HTTP requests, extensibly returning Clojure data
structures for most outputs.
Please use Neotyk's http.async.client here: https://github.com/neotyk/http.async.client

Some knowledge of the underlying Apache `HttpClient` API is necessary for more
advanced usage, such as specifying cookie jars.
Neotyk's library has all of the same features as this one, is more stable,
and is actively maintained!

For real-world code using this library, see the `clj-mql` project:
# Secure Synchronous and Asynchronous HTTP Client #

<https://github.com/rnewman/clj-mql/tree/master>
This is a Clojure wrapper around the Apache Foundation's HTTP client. It
supports synchronous and asynchronous HTTP connections, as well as X509
client certificates.


# Building with Ant #
For information on using the standard synchronous HTTP client, see:

Invoke `ant`, optionally passing `-Dclojure.jar="..."` and `-Dclojure.contrib.jar="..."`.
<https://github.com/diamondap/clj-apache-http/blob/master/README_HTTP.markdown>

Put `clj-apache-http.jar` on your classpath.
For information on using the standard synchronous HTTP client with X509
client certificates, see:

# Building with Leiningen #
<https://github.com/diamondap/clj-apache-http/blob/master/README_X509.markdown>

If you use Leiningen, run `lein uberjar`. This will download the necessary
dependencies and build a single .jar named `clj-apache-http-standalone.jar`.
For information on using the asynchronous client, with or without X509
certificates, see:

You can also refer to `com.twinql.clojure/clj-apache-http "2.2.0"` in Leiningen
or Maven to have the dependency automatically satisfied.

# Loading #

This library defines a function named get, so it's best to require it like
this:

(require ['com.twinql.clojure.http :as 'http])

or in your `(ns)` form:

(:require [com.twinql.clojure.http :as http])


# Functions #

The exported interface consists of the functions `get`, `post`, `put`, `head`,
and `delete`, and the utility function `http-host` (to return a value suitable
for the host parameters, such as `:default-proxy` / `DEFAULT_PROXY`).

All functions take a "uri part" as input -- a URI, a string parsed as a URI, or a map like

{:host "foo.com" :path "/bar/" :port 9000 :query {:x 5} :scheme "https" :fragment "hah"}

and the following keyword arguments:

* `:query` — a query parameter map.
* `:headers` — a map of HTTP headers.
* `:parameters` — a map of values to be passed to `HttpParams.setParameter`.

`post` and `put` additionally have a `:body` argument, which must be an `HttpEntity`.

The result of calling these functions is a map as follows:

* The status code: `:code`
* The reason phrase: `:reason`
* The content (subject to transformation): `:content`
* The Apache `Entity` associated with the request/response pair: `:entity`
* The Apache `HttpClient` used for the request (which allows access to the cookie jar): `:client`
* The response headers (also subject to transformation): `:headers`.

Typically most of these can be ignored; `:code` and `:content` are the most important fields.

Note that minimal processing is applied to these results (you don't pay for
what you don't use).

You can specify the format in which you wish to receive body content and
headers using the `:as` and `:headers-as` keyword arguments. See below for
details.

# Query parameters *

Query parameters (as supplied to the `:query` argument) should be associative:
either a map or a sequence of pairs. Parameters will be processed with
`as-str`. Non-sequential values will also be processed with `as-str`;
sequential values (such as vectors) will be turned into multiple query
parameters, as expected by most HTTP servers. For example:

(encode-query {:foo "bar" :baz ["noo" 5 true] :top {:x 5 :y 7}})
=>
"foo=bar&baz=noo&baz=5&baz=true&top=%7B%3Ax+5%2C+%3Ay+7%7D"


# Examples #

(:content (http/get (java.net.URI. "http://example.com") :as :string))
=>
"<HTML>\r\n<HEAD>\r\n <TITLE>Example Web Page</TITLE>\r\n</HEAD> \r\n<body>…"

(select-keys
(http/get "http://clojure.org/api" :as :stream) [:code :reason :content])
=>
{:content #<EofSensorInputStream org.apache.http.conn.EofSensorInputStream@4ba57633>,
:reason "OK",
:code 200}

(:reason (http/post "http://google.com/search" :query {:q "frobnosticate"}))
=>
"Method Not Allowed"

(:code (http/get "http://google.com/search" :query {:q "frobnosticate"}))
=>
200

(:headers (http/get "http://google.com/search" :query {:q "frobnosticate"} :headers-as :map))
=>
{"Transfer-Encoding" ["chunked"],
"Server" ["gws"],
"Set-Cookie"
["SS=Q0=ZnJvYm5vc3RpY2F0ZQ; path=/search"
["PREF=ID=9174bb4f419f1279:TM=1248028028:LM=1248028028:S=-Nxn6QHqif9SORnQ; expires=Tue, 19-Jul-2011 18:27:08 GMT; path=/; domain=.google.com"]
["NID=24=E9wDEpOrfSFh2bt36RkK7ZIkjH78DeKeA1mulw1A5562byJ2ngJjDPClEHUceb-6ewf7ANSrA-6CrXjUfeHszmv3OM7giddIDfX-RBvtZGYIWI0FbUNbYvoKQXtQRu9S; expires=Mon, 18-Jan-2010 18:27:08 GMT; path=/; domain=.google.com; HttpOnly"]],
"Content-Type" ["text/html; charset=ISO-8859-1"],
"Expires" ["-1"],
"Date" ["Sun, 19 Jul 2009 18:27:08 GMT"],
"Cache-Control" ["private, max-age=0"]}



# Keyword parameters #

You can use `:query`, `:headers`, `:parameters`, `:as`, and `:headers-as`.

The first three are associative. `:as` can be:

* `:identity` (or `nil`), returning the Apache HC entity object,
* `:stream`, returning a stream,
* `:reader`, returning a `Reader`,
* `:json`, which will parse a JSON response body and return the keys of maps as keywords,
* `:json-string-keys`, which leaves maps with string keys,
* or `:string`.

`:headers-as` can be

* `:identity`, returning a `HeaderIterator`,
* `:seq` (or `nil`), returning a sequence of [header, value] pairs,
* `:element-seq`, returning a sequence of [header, `Element[]`] pairs,
* `:map`, returning a map from header name to vector of values, or
* `:element-map`, returning a map from header name to vector of `Element[]`.

Define your own extensions by defining a method on '`entity-as`' that turns an
`HttpEntity` into the appropriate format, or '`headers-as`' that turns a
`HeaderIterator` into the format of your choice.

Each entity transformer receives the entity, the format, and the status code as arguments: you can thus avoid trying to process certain failure responses. For example:

(defmethod http/entity-as :success-json [#^HttpEntity entity as status]
(http/entity-as entity
(if (<= 200 status 299)
:json
:string)
status))


# Apache connection parameters #
You can pass a parameter map to the HTTP functions. This is used to set various
options on the HTTP client.

The keys are long-winded Java constants, but the capability is very useful
(e.g., for proxying). See

<http://hc.apache.org/httpcomponents-client-4.0.1/httpclient/apidocs/org/apache/http/client/params/AllClientPNames.html>

To avoid verbosity, a function `map->params` is provided. This will rename the keys in your input to the appropriate constants.

For example, to issue a `HEAD` request via a proxy:

(http/head "http://github.com/"
:parameters (http/map->params
{:default-proxy (http/http-host :host "localhost" :port 8080)}))

# Optimization #

If you're planning intensive use of HTTP operations, you might want more
control over the connection manager (the component which maintains a pool of
connections).

`clj-apache-http` exposes one utility, `thread-safe-connection-manager`. It can
be called with no arguments, or with a `SchemeRegistry`.

Simply hold on to one of these, then pass it as the value of
`:connection-manager`, as in this example:

(let [ccm (http/thread-safe-connection-manager)]
(println "Result:"
(:code (http/get "http://github.com" :as :string
:connection-manager ccm)))
(http/shutdown-connection-manager ccm))

Don't forget to call `shutdown-connection-manager` afterwards. The
`with-connection-manager` macro does this for you, if your uses will all be
within the same lexical scope:

(http/with-connection-manager [ccm :thread-safe]
(println "Result:"
(:code (http/get "http://github.com" :as :string
:connection-manager ccm))))
<https://github.com/diamondap/clj-apache-http/blob/master/README_ASYNC.markdown>

Loading