Combining Twain and PostgreSQL.Simple

Inspired by the blog post Why I use Twain I am trying to make a simple application which uses Twain and PostreSQL.Simple.

The motivation is that I currently have a couple of websites using Spock, and they leak memory (multiple gigabytes per week), as the author of the blog post above also reports.

As you’ll see in a moment, I’m a beginner in combining libraries/Haskell, so instead of converting my Spock based application directly, I started from the Twain demo and tried to incorporate a database connection pool.

I’m trying to avoid too much boilerplate when running the database queries. This is where I got stuck:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Data.Text hiding (foldr, index, head)
import Data.Text.Lazy as TL hiding (foldr, index, head, Text)
import Data.Text.Lazy.Encoding as TL

import Database.PostgreSQL.Simple
import qualified Database.PostgreSQL.Simple as Pg
import qualified Database.PostgreSQL.Simple.FromRow as Pg
import Data.Pool

import Network.Wai.Handler.Warp (run)
import Web.Twain

import Control.Monad.IO.Class (liftIO)

main :: IO ()
main = do
  pool <- myPool
  putStrLn "mark listening on port 8080 ..."
  run 8080 $ foldr ($) (notFound missing) (routes pool)

routes :: Pool Connection -> [Middleware]
routes pool = [ get "/" (liftIO . withResource pool $ index)
              , get "/echo/:name" echoName
              ]

connectionInfo :: ConnectInfo
connectionInfo = defaultConnectInfo { connectHost = "localhost"
                                    , connectPort = 5432
                                    , connectUser = "lantern"
                                    , connectPassword = "xxxxxx"
                                    , connectDatabase = "lantern"
                                    }

--                                                   max idle time (s)
--                                          subpools |
myPool :: IO (Pool Connection) --                  | |  max connections
myPool = createPool (connect connectionInfo) close 2 60 20

data Count = Count { countInt :: Int } deriving Show

instance FromRow Count where
  fromRow = do
    count <- Pg.field
    return (Count count)

index :: Connection -> IO (ResponderM a)
index db = do
  count <- Pg.query_ db "SELECT COUNT(*) FROM article"
  print count
  return $ send $ html ("Hello World " <> TL.encodeUtf8 (TL.pack (show (countInt (head count)))))

echoName :: ResponderM a
echoName = do
  name <- param "name"
  send $ html $ "Hello " <> name

missing :: ResponderM a
missing = send $ html "Not found…"

This compiles and outputs:

mark listening on port 8080 ...
[Count {countInt = 2372}]

when hit http://localhost:8080/ but the browser shows “Nothing&” rather than the expected “Hello World 2372”.

If I move the liftIO . withResource pool from the routes function and into the index function, on the Pg.query_ line, and pass the pool rather than the db, I get the page content I want. But having that boilerplate for every query is going to drive me bonkers for the real application.

I tried finding Twain examples for inspiration, but only found tutorials without database usage.

Any tips are welcome!

1 Like

Instead of wrapping a ResponderM in IO you can just call IO functions inside a ResponderM using liftIO. This is because ResponderM has an instance for MonadIO.

get "/" (index pool)
index :: Pool Connection ->  ResponderM a
index pool = do
  count <- liftIO $
    withResource pool $ \db -> Pg.query_ db "SELECT COUNT(*) FROM article"
  liftIO $ print count
  send $ html ("Hello World " <> TL.encodeUtf8 (TL.pack (show (countInt (head count)))))

A result is not sent back to the user in the previous code because it wraps the send in IO using return and then back in ResponderM with liftIO. In this code above we directly user send instead.

You could actually also extract the ResponderM a and then use it,

get "/" $ do
  responder <- liftIO . withResource pool $ index
  responder

but that doesn’t really seem like a thing you want to do!

Edit: Another thing you could do to avoid the boilerplate is to wrap your query code like this:

query_ pool sql =
  liftIO $ withResource pool $ \db -> Pg.query_ db sql

then you can write:

  count <- query_ pool "SELECT COUNT(*) FROM article"

pardon if something doesn’t compile, I didn’t try to run this.

That’s what (I tried to describe that) I started by doing, but then I wanted to get rid of the liftIO $ withResource pool $ \db -> boilerplate, and I wanted to get a single db for the entire function.

I think that last ambition is what tripped me up.

Thanks for this hint, that was unclear to me!

That works nicely - I think I got to something similar at some point, but then I wanted a single db for the entire index function and couldn’t crack that nut.

One reason to have one db, instead of getting a “new” one from the pool every time I need it, is for transactions. But maybe I will just wrap those in a big withResource pool, and be done with it.

Thanks for the help!

Makes sense. There are still more options available to reduce the boilerplate and keep the behaviour you want (like creating combinators), but my suggestion is to just start with something that does what you want and after using it for a while figure out how to fix it, if it really ends up annoying you.

1 Like

Now that it’s not 1am I have another Idea - make withResource work for any MonadIO:

withPool :: MonadIO m => Pool a -> (a -> m b) -> m b
withPool pool action = do
  r <- liftIO $ withResource pool (pure . action)
  r

And now you can use it in the routing:

 get "/" (withPool pool index)

And index can have the type Connection -> ResponderM a.

I don’t think withResource can be made to work with arbitrary MonadIO because of the callback. (Being an instance of MonadIO doesn’t mean you can arbitrarily turn an (a -> m b) into an (a -> IO b)) However, in this specific case, you can probably do something like

withPool :: (MonadIO m, MonadCatch m, MonadThrow m) => Pool a  -> (a -> m b) -> m b
withPool pool f = do
  (x, localPool) <- liftIO (takeResource pool)
  -- finally would be nicer, but we can't require a MonadMask instance
  let returnResource = liftIO (putResource localPool x)
  res <- catchAll (f x) (\ex -> returnResource >> throwM ex)
  returnResource
  pure res
2 Likes

Oh I get what you’re saying. I now see that my idea was to essentially try to escape the withResource by returning the resource from the callback, but that’s really not what we want and not a solution. Thank you!

You could use unliftio-pool for a version of withResource that uses a MonadUnliftIO constraint.

4 Likes

Actually @gilmi has one tutorial here: Building a bulletin board using Haskell, twain, sqlite-easy and friends although that is using sqlite.

1 Like

Thanks! - I missed that last sentence when I skimmed through the accompanying blogpost.

Thanks!

I finally got back to this now, and I managed to updated my code above to Data.Pool's newPool function (instead of createPool which is deprecated), by simply changing myPool to:

myPool = newPool (defaultPoolConfig (connect connectionInfo) close 60 20)

So I thought I would start simple and just switch to import UnliftIO.Pool instead of import Data.Pool, so:

import UnliftIO.Pool
-- ...
myPool :: IO (Pool Connection)
myPool = newPool (mkDefaultPoolConfig (connect connectionInfo) close 60 20)

But that doesn’t compile:

app/Main.hs:39:10: error:
    • Couldn't match type ‘PoolConfig Connection’ with ‘Connection’
      Expected: IO (Pool Connection)
        Actual: IO (Pool (PoolConfig Connection))
    • In the expression:
        newPool (mkDefaultPoolConfig (connect connectionInfo) close 60 20)
  [...]

I thought I’d try this simple change before trying to do the actual change I want (to have the same pool for all requests), but I am already stuck :slight_smile:

I guess it’s something obvious/simple, but I can’t see it. I tried a couple of things, but still stumped.

Disclaimer: I haven’t used this library or Pool. But from a look a hoogle I think you need:

myPool = do 
    c <- mkDefaultPoolConfig ...
    newPool c 
1 Like

Thanks! That was it - I usually am able to spot that pattern, but not this time, for some reason.

Great! I will move on to trying to rearrange things to have one pool.

Thanks for all the help so far!

Thanks to your suggestions and help, I’m here now:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Data.Text hiding (foldr, index, head)
import Data.Text.Lazy as TL hiding (foldr, index, head, Text)
import Data.Text.Lazy.Encoding as TL

import Database.PostgreSQL.Simple
import qualified Database.PostgreSQL.Simple as Pg
import qualified Database.PostgreSQL.Simple.FromRow as Pg
import UnliftIO.Pool

import Network.Wai.Handler.Warp (run)
import Web.Twain

import Control.Monad.IO.Class (liftIO)

main :: IO ()
main = do
  config <- mkDefaultPoolConfig (connect connectionInfo) close 60 20
  pool <- newPool config
  putStrLn "mark listening on port 8080 ..."
  run 8080 $ foldr ($) (notFound missing) (routes pool)

routes :: Pool Connection -> [Middleware]
routes pool = [ get "/" (index pool)
              ]

connectionInfo :: ConnectInfo
connectionInfo = defaultConnectInfo { connectHost = "virgil.koldfront.dk"
                                    , connectPort = 5432
                                    , connectUser = "XXXX"
                                    , connectPassword = "XXXX"
                                    , connectDatabase = "XXXX"
                                    }

poolQuery_ pool sql = liftIO $ withResource pool $ \db -> Pg.query_ db sql

newtype Count = Count { countInt :: Int } deriving Show

instance FromRow Count where
  fromRow = do Count <$> Pg.field

index :: Pool Connection -> ResponderM a
index pool = do
  count <- poolQuery_ pool "SELECT COUNT(*) FROM article"
  liftIO $ print count
  send $ html ("Hello World " <> TL.encodeUtf8 (TL.pack (show (countInt (head count)))))

missing :: ResponderM a
missing = send $ html "Not found..."

and pretty happy with it - thanks!

2 Likes