FFI Questions (Wrapping C++ code, and callbacks)

Learn
1

So, I’m not new to Haskell, but I’ve never worked much with the C FFI. I figured a fun place to start would be adding some really basic Botan.TLS bindings to the botan-bindings package, just to see how rough it would be (to be clear, this isn’t in any sort of shape to be reviewed for inclusion, and I haven’t talked with @ApothecaLabs at all to even see if he’s interested, I’m just doing it to learn, and also my C++ experience is a few decades out of date…).

I wound up with a TLS echo server that works (specific diff against botan is here, initially modeled after the callerback example on the wiki). But, I wound up with a few questions…

1 - How can I pass a (Ptr CString) to a C function? My C bindings are wrapping C++ functions that can raise exceptions, so I’d like to do something like this:

void *allocate(char **errmsg) {
  try {
    return new Server();
  } catch(std::exception &exc) {
    *errmsg = new char[strlen(exc.what())+1];
    strncpy(errmsg, exc.what(), strlen(exc.what())+1);
    return nullptr;
  }
}

And then call that giving some sort of errmsg :: Ptr CString, but I don’t have any idea how to allocate that (or how to dereference it after the call). So instead I’m allocating a blank error message string using withCString, giving it a length of 1024, and letting the callback copy the error message into that buffer. I assume there’s a good way to do this, but I’m having no luck figuring it out.

2 - Should I be able to do Network.Socket IO in a haskell callback being called from C++?

Botan.TLS has you set up a Server object with a few callbacks. One of them is emitData, which takes a pointer and a length, and you’re supposed to send that data to the connected client. Easy enough, there’s a Network.Socket.sendBuf that does exactly that, but when I call it from a callback, the resulting “bytes written” was always 0. I’m guessing that the async IO doesn’t run while C callbacks are being called? This was in ghci, so maybe a fully compiled and threaded executable would have worked better, but I’d like to not have something totally broken in ghci anyhow. I kludged around it by copying the Ptr/len to a ByteString and throwing that into a queue to be handled after the callback, which is fine but I wanted to make sure I wasn’t missing something obvious.

3 - How do I clean up?

I think the normal pattern for cleaning up something that allocates is to wrap the Ptr into a ForeignPtr that tracks the deallocate function. But that function itself is a FunPtr, and I think those need to be freed using freeHaskellFunPtr? So maybe all the allocated data and callback functions just need to be freed in a normal bracket pattern, and that’s fine, but I’m still confused about how a ForeignPtr can ever be safely used. Or am I misunderstanding a whole lot of stuff there?

Thanks for any help!

2
  1. I definitely need to finish memalloc to make this sort of thing easier, but for now I would either, depending on your needs:

Note that mallocByteString is really just a specialized / optimized mallocForeignPtr, which itself has some helpful notes:

mallocForeignPtr is equivalent to

   do { p <- malloc; newForeignPtr finalizerFree p }

And mallocBytes is literally just a wrapped up malloc, so it is really all the same function just wrapped up differently.

Without knowing more specifics, I’d probably go with mallocByteString:

  • it gives you a ForeignPtr that you can use withForeignPtr to access the Ptrfor C code
  • it will be eventually freed when you are no longer using it.
  • if you know the buffer is only temporary, you can deliberately trigger the garbage collection immediately using finalizeForeignPtr

  1. I’m guessing that the async IO doesn’t run while C callbacks are being called?

Almost certainly. GHCi is single-threaded by default, and you have to turn on multi-threading, and there are still some subtle differences from compiled code.

  1. Already covered in 1 - you want a ForeignPtr and to use finalizeForeignPtr to trigger cleanup manually, or you can just let it get garbage collected later.

Now, go forth and allocate!

3

Working through your reply, but I think the Alloc package was what I wanted. I’m doing this and it’s working, but I’m not sure it’s sane:

      alloca $ \errmsgptr -> do
        server <- serverNew cKeyPath cCrtPath emitW recordW alertW errmsgptr
        if server == nullPtr
          then do
            errstr <- peek errmsgptr
            clone <- peekCString errstr
            free errstr
            pure $ Left clone
          else pure $ Right server

and the associated C wrapper is now

void* tls_server_new(..., char **errmsg)
{
    try
    {
         ...
    }
    catch(std::exception& exc)
    {
        int buflen = strlen(exc.what())+1;
        *errmsg = new char[buflen];
        strncpy(*errmsg, exc.what(), buflen);
        return nullptr;
    }
}

So the C++ code is only allocating the error message when it needs to, and the Haskell code only has to allocate the size of a pointer (which I assume gets optimized into bumping the stack enough to have an int? or maybe it’s a heap-allocated int, which is gross but less annoying than an entire arbitrary-sized string).

I still feel like I have one more pointer on the Haskell side than I strictly need, because calling alloca to give me a pointer to a pointer seems wrong, but maybe that’s just how it is? I’m getting my error messages successfully and this is a pattern that I think won’t drive me crazy if I apply it to all the other places that are wrapping C++ code.

I’ll look into the rest of your reply next, thanks a ton!

4

Makes sense. Does it seem sane to append items to a TQueue (STM) in a callback? Do you know of any sort of list of “safe things to call” from within the FFI?

Yeah, I figured out part of my problem, which was that I assumed I had to expose and use my free function like this:

foreign import ccall "botan-tls-wrapper.hpp tls_server_free"
  serverFree :: TlsServer -> IO ()

foreign import ccall "wrapper"
  wrapFree :: (TlsServer -> IO ()) -> IO (FunPtr (TlsServer -> IO()))

-- and then using in a function like
foo = do
 finalizer <- wrapFree serverFree
 tlsForeignPtr <- newForeignPtr finalizer server

but those first two lines are better written as a single item, which can be directly added as finalizer like this:

foreign import ccall "botan-tls-wrapper.hpp &tls_server_free"
  serverFreeFun :: FunPtr (TlsServer -> IO ())

foo = do
 tlsForeignPtr <- newForeignPtr serverFreeFun server

So that’s cool, I can make top-level functions FunPtr when I need to. I’m still not sure about the dynamically-wrapped functions though. I’m defining “wrapper” calls like this:

type EmitData = Ptr Word8 -> Word64 -> IO ()

type RecordReceived = Word64 -> Ptr Word8 -> Word64 -> IO ()

type Alert = Ptr () -> IO ()

-- Wrap an EmitData function into something C++ can call
foreign import ccall "wrapper"
  wrapEmitData :: EmitData -> IO (FunPtr EmitData)

-- Wrap a RecordReceived function into something C++ can call
foreign import ccall "wrapper"
  wrapRecordReceived :: RecordReceived -> IO (FunPtr RecordReceived)

-- Wrap an Alert function into something C++ can call
foreign import ccall "wrapper"
  wrapAlert :: Alert -> IO (FunPtr Alert)

and then when I make a new Botan.TLS object I wrap up Haskell functions and hand them off to the C++ code like this:

emitData :: TQueue (Maybe ByteString) -> EmitData
recordReceived :: (TQueue (Word64, ByteString)) -> RecordReceived
alert :: Alert

foo = do
  emitW <- wrapEmitData $ emitData outbox
  recordW <- wrapRecordReceived $ recordReceived inbox
  alertW <- wrapAlert alert
  ...
  server <- serverNew cKeyPath cCrtPath emitW recordW alertW errmsgptr

But the FunPtr docs say that FunPtrs allocated that way need to be freed up with calls to freeHaskellFunPtr :: FunPtr a → IO (), and that doesn’t seem like something I could easily add as a finalizer function. I’ll keep poking away at it though :slight_smile:

5

alloca is very powerful when used properly, but you are using it in a scary dangerous way. Why are you allocating a pointer to a pointer? Are you allocating a buffer in C++ and passing ownership if it to Haskell and freeing it in Haskell with free? You should not be doing that in C, let alone C++ with its hidden constructor shenanigans.

What you should be doing is allocaBytes 4096 $ \ (errmsgptr :: Ptr CChar) -> ... a buffer that is owned by Haskell, pass that in to a void * tls_server_new(..., char * errmsg, size_t errmsg_count) using server <- serverNew ... errmsgptr 4096 and then the C++ fills the buffer that you gave it.

You never want to transfer ownership over FFI, you want to copy data.

6

Okay, that’s what I had been doing, it just bugged me because I don’t actually know how big the error message will be, or even if there will be one. Seemed like skipping the allocation except for when it’s necessary would be the right approach, but the Haskell side sure does seem weird. I can go back to the “always allocate” approach if that’s the sane way to do things :smiley:

7

The sooner you can return control back to foreign code in a callback, the better. Best practice is to copy out the necessary data from the callback context, send that data to be processed / handled a few microseconds later on another thread, and immediately return control. This doesn’t really work if the callback is blocking by expecting a result, but that is rare

Yeah this is why its best to use the mallocBytes with a ForeignPtr and letting C++ fill it (instead of allocating it in C++ and returning it), because dealing with finalizers is complicated :laughing:

8

Don’t worry about it - just allocate a buffer of 4096 bytes with allocaBytes - 4kb is nothing on a machine with gigabytes to spare. Its common in C to just allocate a 4kb buffer on the stack for handling IO and error messages, and most OS’s have like a 8kb or 16kb buffer for printf so like yeah, that huge buffer gets used every time you print a single “\n”. So don’t feel bad, alloca allocates memory from a special pool (its ‘on the stack’) that is very fast for exactly this sort of thing because it knows it is going to be immediately released.

Or, allocate the buffer at the beginning of your Haskell program, and reuse it every time you do an operation that might result in an error message. Then it doesn’t matter how big it is, you’re reusing it.

9

Thanks for all the help, I think I have a better clue about what I’m doing anyhow, or at least I have a good list of mistakes to correct.

Is there interest in trying to expose the Botan.TLS stuff in your botan package? I know what I have right now isn’t at all ready for review or inclusion, but do you think it’s something you’d be interested in at some point? Right now I’m trying to put together enough functionality to make a warp-tls type package that uses botan, so my needs are pretty minimal. Not sure what it would take to be worth merging with your work :slight_smile:

Oh, that’s good to know. Thanks!

10

Unofficial bindings (unofficial in the sense of, not published by Randombit) should probably be a separate package for security reasons, but botan-tls is perfectly reasonable. Also, that way it won’t conflict if Botan gets official TLS C FFI bindings that can be included in the official package.

Otherwise, hurrah and welcome! I appreciate the help on the botan project, TLS bindings are a low-urgency, high-priority item on the todo list, so even unofficial bindings is a huge win!