The HTTP client for Scala that you always wanted!
import com.softwaremill.sttp._
val sort: Option[String] = None
val query = "http language:scala"
// the `query` parameter is automatically url-encoded
// `sort` is removed, as the value is not defined
val request = sttp.get(uri"https://api.github.com/search/repositories?q=$query&sort=$sort")
implicit val backend = HttpURLConnectionBackend()
val response = request.send()
// response.header(...): Option[String]
println(response.header("Content-Length"))
// response.unsafeBody: by default read into a String
println(response.unsafeBody)
- provide a simple, discoverable, no-surprises, reasonably type-safe API for making HTTP requests and reading responses
- separate definition of a request from request execution
- provide immutable, easily modifiable data structures for requests and responses
- support multiple execution backends, both synchronous and asynchronous
- provide support for backend-specific request/response streaming
- minimum dependencies
See also the introduction to sttp and sttp streaming & URI interpolators blogs.
- implement a full HTTP client. Instead, sttp wraps existing HTTP clients, providing a consistent, programmer-friendly API. All network-related concerns such as sending the requests, connection pooling, receiving responses are delegated to the chosen backend
- provide ultimate flexibility in defining the request. While it's possible to define most valid HTTP requests, e.g. some of the less common body chunking approaches aren't available
- immutable request builder which doesn't impose any order in which request parameters need to be specified. Such an approach allows defining partial requests with common cookies/headers/options, which can later be specialized using a specific URI and HTTP method.
- support for multiple backends, both synchronous and asynchronous, with backend-specific streaming support
- URI interpolator with context-aware escaping, optional parameters support and parameter collections
If you are an Ammonite user, you can quickly start experimenting with sttp by copy-pasting the following:
import $ivy.`com.softwaremill.sttp::core:0.0.15`
import com.softwaremill.sttp._
implicit val backend = HttpURLConnectionBackend()
sttp.get(uri"http://httpbin.org/ip").send()
SBT dependency:
"com.softwaremill.sttp" %% "core" % "0.0.15"
sttp
is available for Scala 2.11 and 2.12, and requires Java 7 if using an OkHttp
based backend, or Java 8 otherwise. The core
module has no transitive dependencies.
If you'd like to use an alternate backend, see below for additional instructions.
First, import:
import com.softwaremill.sttp._
This brings into scope sttp
, the starting request (it's an empty request
with the Accept-Encoding: gzip, defalte
header added). This request can
be customised, each time yielding a new, immutable request description
(unless a mutable body is set on the request, such as a byte array).
For example, we can set a cookie, string-body and specify that this should
be a POST
request to a given URI:
val request = sttp
.cookie("login", "me")
.body("This is a test")
.post(uri"http://endpoint.com/secret")
The request parameters (headers, cookies, body etc.) can be specified in any order. There's a lot of ways in which you can customize a request: just explore the API. And more will be added!
You can create a request description without knowing how it will be sent.
But to send a request, you will need a backend. A default, synchronous backend
based on Java's HttpURLConnection
is provided out-of-the box. An implicit
value of type SttpBackend
needs to be in scope to invoke the send()
on the
request:
implicit val backend = HttpURLConnectionBackend()
val response: Response[String] = request.send()
By default the response body is read into a utf-8 string. How the response body
is handled is also part of the request description. The body can be ignore
(.response(ignore)
), read into a sequence of parameters
(.response(asParams)
), mapped (.mapResponse
) and more; some backends also
support request & response streaming.
The default backend doesn't wrap the response into any container, but other
asynchronous backends might do so. The type parameter in the Response[_]
type specifies the type of the body.
Using the URI interpolator it's possible to conveniently create Uri
instances, which can then be used to specify request endpoints, for example:
import com.softwaremill.sttp._
val user = "Mary Smith"
val filter = "programming languages"
val endpoint: Uri = uri"http://example.com/$user/skills?filter=$filter"
Any values embedded in the URI will be URL-encoded, taking into account the
context (e.g., the whitespace in user
will be %-encoded as %20D
, while the
whitespace in filter
will be query-encoded as +
).
The possibilities of the interpolator don't end here. Other supported features:
- parameters can have optional values: if the value of a parameter is
None
, it will be removed - maps, sequences of tuples and sequences of values can be embedded in the query
part. They will be expanded into query parameters. Maps and sequences of tuples
can also contain optional values, for which mappings will be removed
if
None
. - optional values in the host part will be expanded to a subdomain if
Some
, removed ifNone
- sequences in the host part will be expanded to a subdomain sequence
- if a string containing the protocol is embedded as the very beginning, it will
not be escaped, allowing to embed entire addresses as prefixes, e.g.:
uri"$endpoint/login"
, whereval endpoint = "http://example.com/api"
.
A fully-featured example:
import com.softwaremill.sttp._
val secure = true
val scheme = if (secure) "https" else "http"
val subdomains = List("sub1", "sub2")
val vx = Some("y z")
val params = Map("a" -> 1, "b" -> 2)
val jumpTo = Some("section2")
uri"$scheme://$subdomains.example.com?x=$vx&$params#$jumpTo"
// generates:
// https://sub1.sub2.example.com?x=y+z&a=1&b=2#section2
In case of most backends, you should only instantiate a backend once per application, as a backend typically allocates resources such as thread or connection pools.
When ending the application, make sure to call backend.close()
, which will
free up resources used by the backend (if any). The close process might be
asynchronous, that is it can complete after the close()
method returns.
Note that only resources allocated by the backends are freed. For example,
if you use the AkkaHttpBackend()
the close()
method will terminate the
underlying actor system. However, if you have provided an existing actor system
upon backend creation (AkkaHttpBackend.usingActorSystem
), the close()
method will be a no-op.
Class | Result wrapper | Supported stream type |
---|---|---|
HttpURLConnectionBackend |
None (Id ) |
- |
AkkaHttpBackend |
scala.concurrent.Future |
akka.stream.scaladsl.Source[ByteString, Any] |
AsyncHttpClientFutureBackend |
scala.concurrent.Future |
- |
AsyncHttpClientScalazBackend |
scalaz.concurrent.Task |
- |
AsyncHttpClientMonixBackend |
monix.eval.Task |
monix.reactive.Observable[ByteBuffer] |
AsyncHttpClientCatsBackend |
F[_]: cats.effect.Async |
- |
AsyncHttpClientFs2Backend |
F[_]: cats.effect.Async |
fs2.Stream[F, ByteBuffer] |
OkHttpSyncBackend |
None (Id ) |
- |
OkHttpFutureBackend |
scala.concurrent.Future |
- |
OkHttpMonixBackend |
monix.eval.Task |
monix.reactive.Observable[ByteBuffer] |
The default synchronous backend. Sending a request returns a response wrapped in the identity type constructor, which is equivalent to no wrapper at all.
To use, add an implicit value:
implicit val sttpBackend = HttpURLConnectionBackend()
To use, add the following dependency to your project:
"com.softwaremill.sttp" %% "akka-http-backend" % "0.0.15"
This backend depends on akka-http.
A fully asynchronous backend. Sending a request returns a response wrapped
in a Future
.
Next you'll need to add an implicit value:
implicit val sttpBackend = AkkaHttpBackend()
// or, if you'd like to use an existing actor system:
implicit val sttpBackend = AkkaHttpBackend.usingActorSystem(actorSystem)
This backend supports sending and receiving
akka-streams
streams of type akka.stream.scaladsl.Source[ByteString, Any]
.
To set the request body as a stream:
import com.softwaremill.sttp._
import com.softwaremill.sttp.akkahttp._
import akka.stream.scaladsl.Source
import akka.util.ByteString
val source: Source[ByteString, Any] = ...
sttp
.streamBody(source)
.post(uri"...")
To receive the response body as a stream:
import com.softwaremill.sttp._
import com.softwaremill.sttp.akkahttp._
import akka.stream.scaladsl.Source
import akka.util.ByteString
implicit val sttpBackend = AkkaHttpBackend()
val response: Future[Response[Source[ByteString, Any]]] =
sttp
.post(uri"...")
.response(asStream[Source[ByteString, Any]])
.send()
To use, add the following dependency to your project:
"com.softwaremill.sttp" %% "async-http-client-backend-future" % "0.0.15"
// or
"com.softwaremill.sttp" %% "async-http-client-backend-scalaz" % "0.0.15"
// or
"com.softwaremill.sttp" %% "async-http-client-backend-monix" % "0.0.15"
// or
"com.softwaremill.sttp" %% "async-http-client-backend-cats" % "0.0.15"
This backend depends on async-http-client. A fully asynchronous backend, which uses Netty behind the scenes.
The responses are wrapped depending on the dependency chosen in either a:
- standard Scala
Future
- Scalaz
Task
. There's a transitive dependency onscalaz-concurrent
. - Monix
Task
. There's a transitive dependency onmonix-eval
. - Any type implementing the Cats Effect
Async
typeclass, such ascats.effect.IO
. There's a transitive dependency oncats-effect
.
Next you'll need to add an implicit value:
implicit val sttpBackend = AsyncHttpClientFutureBackend()
// or, if you're using the scalaz version:
implicit val sttpBackend = AsyncHttpClientScalazBackend()
// or, if you're using the monix version:
implicit val sttpBackend = AsyncHttpClientMonixBackend()
// or, if you're using the cats effect version:
implicit val sttpBackend = AsyncHttpClientCatsBackend[cats.effect.IO]()
// or, if you'd like to use custom configuration:
implicit val sttpBackend = AsyncHttpClientFutureBackend.usingConfig(asyncHttpClientConfig)
// or, if you'd like to instantiate the AsyncHttpClient yourself:
implicit val sttpBackend = AsyncHttpClientFutureBackend.usingClient(asyncHttpClient)
The Monix backend supports streaming (as both Monix and Async Http Client
support reactive streams Publisher
s out of the box). The type of
supported streams in this case is Observable[ByteBuffer]
. That is, you can
set such an observable as a request body:
import com.softwaremill.sttp._
import java.nio.ByteBuffer
import monix.reactive.Observable
val obs: Observable[ByteBuffer] = ...
sttp
.streamBody(obs)
.post(uri"...")
And receive responses as an observable stream:
import com.softwaremill.sttp._
import com.softwaremill.sttp.asynchttpclient.monix._
import java.nio.ByteBuffer
import monix.eval.Task
import monix.reactive.Observable
implicit val sttpBackend = AsyncHttpClientMonixBackend()
val response: Task[Response[Observable[ByteBuffer]]] =
sttp
.post(uri"...")
.response(asStream[Observable[ByteBuffer]])
.send()
It's also possible to use fs2s streams for sending request & receiving responses.
To use, add the following dependency to your project:
"com.softwaremill.sttp" %% "okhttp-backend" % "0.0.15"
// or, for the monix version:
"com.softwaremill.sttp" %% "okhttp-backend-monix" % "0.0.15"
This backend depends on OkHttp, and offers:
- a synchronous backend:
OkHttpSyncBackend
- an asynchronous,
Future
-based backend:OkHttpFutureBackend
- an asynchronous, Monix-
Task
-based backend:OkHttpMonixBackend
OkHttp fully supports HTTP/2.
It is also entirely possible to write your own backend (if so, please consider
contributing!) or wrapping an existing one. You can even write completely
generic wrappers for any delegate backend, as each backend comes equipped
with a monad for the response type. This brings the possibility to map
and
flatMap
over responses.
Possible use-cases for wrapper-backend include:
- logging
- capturing metrics
- request signing (transforming the request before sending it to the delegate)
To pass some context to wrapper-backends, requests can be tagged. Each
RequestT
instance contains a tags: Map[String, Any]
field. This is unused
by http, but can be used e.g. to pass a metric name or logging context.
JSON encoding of bodies and decoding of responses can be handled using
Circe by the circe
module. To use
add the following dependency to your project:
"com.softwaremill.sttp" %% "circe" % "0.0.15"
This module adds a method to the request and a function that can be given to a request to decode the response to a specific object.
import com.softwaremill.sttp._
import com.softwaremill.sttp.circe._
implicit val backend = HttpURLConnectionBackend()
// Assume that there is an implicit circe encoder in scope
// for the request Payload, and a decoder for the Response
val requestPayload: Payload = ???
val response: Either[io.circe.Error, Response] =
sttp
.post(uri"...")
.body(requestPayload)
.response(asJson[Response])
.send()
All request descriptions have type RequestT[U, T, S]
(T as in Template).
If this looks a bit complex, don't worry, what the three type parameters stand
for is the only thing you'll hopefully have to remember when using the API!
Going one-by-one:
U[_]
specifies if the request method and URL are specified. Using the API, this can be eithertype Empty[X] = None
, meaning that the request has neither a method nor an URI. Or, it can betype Id[X] = X
(type-level identity), meaning that the request has both a method and an URI specified. Only requests with a specified URI & method can be sent.T
specifies the type to which the response will be read. By default, this isString
. But it can also be e.g.Array[Byte]
orUnit
, if the response should be ignored. Response body handling can be changed by calling the.response
method. With backends which support streaming, this can also be a supported stream type.S
specifies the stream type that this request uses. Most of the time this will beNothing
, meaning that this request does not send a streaming body or receive a streaming response. So most of the times you can just ignore that parameter. But, if you are using a streaming backend and want to send/receive a stream, the.streamBody
orresponse(asStream[S])
will change the type parameter.
There are two type aliases for the request template that are used:
type Request[T, S] = RequestT[Id, T, S]
. A sendable request.type PartialRequest[T, S] = RequestT[Empty, T, S]
Sttp supports read and connection timeouts:
- Connection timeout - can be set globally (30 seconds by default)
- Read timeout - can be set per request (1 minute by default)
How to use:
import com.softwaremill.sttp._
import scala.concurrent.duration._
// all backends provide a constructor that allows users to specify connection timeout
implicit val backend = HttpURLConnectionBackend(connectionTimeout = 1.minute)
sttp
.get(uri"...")
.readTimeout(5.minutes) // or Duration.Inf to turn read timeout off
.send()
SSL handling can be customized (or disabled) when creating a backend and is backend-specific.
Depending on the underlying backend's client, you can customize SSL settings as follows:
HttpUrlConnectionBackend
: when creating the backend, specify thecustomizeConnection: HttpURLConnection => Unit
parameter, and set the hostname verifier & SSL socket factory as required- akka-http: when creating the backend, specify the
customHttpsContext: Option[HttpsConnectionContext]
parameter. See akka-http docs - async-http-client: create a custom client and use the
setSSLContext
method - OkHttp: create a custom client modifying the SSL settings as described on the wiki
- the encoding for
String
s defaults toutf-8
. - unless explicitly specified, the
Content-Type
defaults to:text/plain
for textapplication/x-www-form-urlencoded
for form datamultipart/form-data
for multipart form dataapplication/octet-stream
for everything else (binary)
Take a look at the open issues and pick a task you'd like to work on!