Hello, Because the company I’m working for is loo...

# http4k

[Paul Reijbroek]

Hello,

Because the company I’m working for is looking to do some operations on CloudWatch that are not supported (yet), I’m working on a PR to include a new CloudWatch module for http4k-connect to support some of those operations. I’ve written functionality in particular for alarms, metric data and metric streams (considering these look like the most common use cases and/or are the operations we need to have), but I’m running into a couple of issues for the fake implementations and the tests. Please see 🧵 for a list of the issues I’m running into. What would be a good approach for them?

[Paul Reijbroek]

Issues with the fakes:

• For metric data and listing metrics, what would be a good way to store and retrieve them? In particular for the Get operations, it seems somewhat difficult to evaluate validity of MathExpressions and MetricDataQueries and very difficult to evaluate them over the provided metrics. What would be an acceptable, simple implementation for the fakes? Or would it be alright not to provide it for metric data, like not all functionality for fakes is provided in the cognito module?
• What validation is expected in the fakes? Should the limits that Amazon itself places on services and data be implemented (e.g. at most 50 tags for any resource) in the fakes, or should we not care about that?
• What is an acceptable way of writing fakes where NextToken is used? Can we actually use it to derive some logic (e.g. always return it with an int value to determine the page) in the fake to support the pagination functionality NextToken normally provides? Or is there a different preferred solution?
Issues with the tests:

• Because I have no fake implementation for metric data, I cannot write tests for them either (because the tests for the fakes would fail). What would be the best approach to ensure there are tests for these functions?
• For testing metric streams, we need an existing Firehose delivery stream. I see that no test dependency exists anywhere between two different AWS service modules. I assume we want to keep it that way. With that in mind, how would I ensure an existing delivery stream exists (or how can I create on) for my tests? I need to know both for the actual implementation and for the fakes, so the tests can work for both.
I may run into more issues, but these are the main ones so far.

[James Richardson]

In general I'd say fakes are a great place to express constraints that you know about services.
Don't support more than n things? Sure great to include it.
They also benefit from being consistent - if the service implements pagination, then it would be great to support that too. Fake services can have state - they only need enough state to support your use case though. Or they can get or be given the state from the test that constructed it - if you're running in-process.
The results that a fake returns don't need to be faithful implementations of the service capability though, only the service behaviour and responses. The fake could accept any math expression or metric data query, and based on its set-up return any response, it doesn't have to actually evaluate these things - unless.you want it to.

[dave]

agree with @James Richardson here. The data has to be valid to be marshalled, but doesn't really need to express any logic beyond that - this would just complicate the implementation. We have a connect Storage interface which can be used to hold any state required in memory/on-disk/redis etc, but we generally beware of putting too much effort into the fake implementations- they should be pretty dumb by default.

For pagination, we have tended to just not really worry about it and return all the data in one shot, although it would be fairly simple to just use a list with a key that can be partitioned to provide multiple pages. once again though - this would complicate the fake.

[Paul Reijbroek]

Thanks for the input @James Richardson and @dave. I’ll take this into account for implementing the fakes for the metric data. That should help for most of the issues I have.

The one issue remaining is the dependency on an existing Firehouse delivery stream for creating metric streams. What would be the best solution for that one?

An alternative could be that I leave the metric streams out for now. That is not part of the functionality I need at the company I work for, but since it looked like a common use case I thought I’d include it. It would then just be alarms and metric data

[James Richardson]

Well, a fake wouldn't need that though, right? It just needs a source of events, which again can just call back to the test class to ask it what new events have arrived..

[Paul Reijbroek]

You’re right when talking about the tests for the fakes. I can basically ignore it there. The tests (supposedly) also run on the main code which do not use the fakes, so that’s a struggle

[James Richardson]

Everything has a fake component though... so your code that connects to cloud watch, can be connected to a fake cloudwatch.. but code that connects to a firehose can connect to a fake one too. You can't always have the same tests against real and fake things, although it is nice, because it's hard or expensive to set some things up, and because it's hard to make real things fail in the way you might want. The "main code" should be able to be configured to talk to a fake environment - how can it know the difference?

[Paul Reijbroek]

I should phrase the question differently, without regards to the fakes and the real imlementations: I need to create a Firehose Delivery Stream (whichever kind it would be), to be able to create a metric stream. The easiest way to do this is to add a test dependency on the http4k-connect Firehose module in the CloudWatch module. However, I see none such dependencies exist in the library (different specific connect services having dependencies on other connect services), and my guess is that it would be desirable to keep it that way. If that is the case, how would I make sure such a delivery stream is there? What would be a good approach then?

[James Richardson]

Ah OK, yes creating dependencies is better avoided. Maybe it would be ok to have another module? http4k-connect-cloudwatch-streaming and that could have deps on both the cloudwatch and the firehose ? That way if you don't use the streaming things, you don't need it.

[Paul Reijbroek]

That could be a good solution. I’ll await some more feedback on that. In the meantime I’ve created a PR without the metric streams.