Test driven software development is built into the fundamentals in the GO
programming language.
The Ory Open Source Software project Dockertest began with the aim to simplify
database testing and reduce the complexity in building unit tests.
In this blogpost, I want to introduce Dockertest, and highlight how its
simplicity and utility in building cloud native database applications has made
it a widespread tool among developers.
What is Ory Dockertest?
In today's cloud native software development, many engineers choose to use
containerization software such Docker to test,
deploy, and ship database applications.
And even though the whole development process has become much easier with
Docker, there are still cases that are hard to handle.
One such case is when you want to test integration of services with databases.
When working with many mock database systems testing can be cumbersome because
the database abstraction layer is often tied to one database type. Using an API
abstraction simplifies the process. But for small changes to the table structure
you need to work on the mocks or deal with other ramifications in the API.
You have the problem that the mock is not a real database. You also have to
rewrite the mock database and unit tests, when you make changes to your
configuration.
This is where Docker and Ory Dockertest can help you create real databases for
your tests. You dont have to set up a whole database infrastructure.
Dockertest automates your interactions with
the Docker API. This lets you test your services database integration in a
simple and straightforward way. You configure your specific test setup, create
the databases you want to test, ping to ensure connection and after testing
close and dispose of the database again.
In Dockertest a pool represents a
connection to the Docker API and is used to handle docker images and resources
represent the docker containers. You can find examples for all common databases
on github.com/ory/Dockertest.
How Dockertest is helping keeping Covid in check
The (Google/Apple) Exposure Notification (GAEN) system was developed by both
companies to help provide an effective framework for tracing Covid19 infections
with an emphasis on privacy. Goverments worldwide use it to provide their
citizens with a means of tracing infections or rather notifiying them of
exposure to the virus. Since most people either have an Apple or Android device
this approach covers a broad chunk of the population.
Now what does this have to do with
Dockertest?
Google's
open source reference implementation
of an notification server uses Dockertest
for
database testing.
It is amazing to see a tool that was written to ease the development process at
Ory is used by big tech organisations in the most impactful health operation in
modern times. It shows what a reliable and essential tool
Dockertest has become over the years.
Here you can see the full code of the implementation:
func NewTestDatabaseWithConfig(tb testing.TB) (*DB, *Config) {
tb.Helper()
if testing.Short() {
tb.Skipf("🚧 Skipping database tests (short!")
}
if skip, _ := strconv.ParseBool(os.Getenv("SKIP_DATABASE_TESTS")); skip {
tb.Skipf("🚧 Skipping database tests (SKIP_DATABASE_TESTS is set)!")
}
ctx := context.Background()
pool, err := Dockertest.NewPool("")
if err != nil {
tb.Fatalf("failed to create Docker pool: %s", err)
}
dbname, username, password := "en-server", "my-username", "abcd1234"
container, err := pool.RunWithOptions(&Dockertest.RunOptions{
Repository: "postgres",
Tag: "12-alpine",
Env: []string{
"LANG=C",
"POSTGRES_DB=" + dbname,
"POSTGRES_USER=" + username,
"POSTGRES_PASSWORD=" + password,
},
})
if err != nil {
tb.Fatalf("failed to start postgres container: %s", err)
}
tb.Cleanup(func() {
if err := pool.Purge(container); err != nil {
tb.Fatalf("failed to cleanup postgres container: %s", err)
}
})
host := container.Container.NetworkSettings.IPAddress
runtime.GOOS == {
host = net.JoinHostPort(container.GetBoundIP(), container.GetPort())
}
connURL := &url.URL{
Scheme: ,
User: url.UserPassword(username, password),
Host: host,
Path: dbname,
}
q := connURL.Query()
q.Add(, )
connURL.RawQuery = q.Encode()
time.Sleep( * time.Second)
b, err := retry.NewFibonacci( * time.Millisecond)
err != {
tb.Fatalf(, err)
}
b = retry.WithMaxRetries(, b)
b = retry.WithCappedDuration(*time.Second, b)
dbpool *pgxpool.Pool
err := retry.Do(ctx, b, {
err
dbpool, err = pgxpool.Connect(ctx, connURL.String())
err != {
retry.RetryableError(err)
}
}); err != {
tb.Fatalf(, err)
}
err := dbMigrate(connURL.String()); err != {
tb.Fatalf(, err)
}
db := &DB{Pool: dbpool}
tb.Cleanup( {
db.Close(context.Background())
})
db, &Config{
Name: dbname,
User: username,
Host: container.GetBoundIP(),
Port: container.GetPort(),
SSLMode: ,
Password: password,
}
}
Health agencies around the world use this reference implementation of the
Exposure Notifications API to build a backend to their notification apps. The
server receives and validates the coded exposure keys from positive cases and
keys shared between mobile devices to determine if two devices were near each
other. It then stores the keys in a database and if there is a corresponding
match on a user's device, the app notifies the user that they have been exposed
to a positive case.
You can read more in depth about the server on
Googles github documentation.
Open-source software drives innovative development
The whole GAEN project is open-source, as is
Dockertest. The whole concept of contact
tracing or exposure notification works great with open source, since it cuts
down complexity of the whole operation. Imagine how much slower and inconsistent
it would be if every country tried to build this software themselves. The sheer
scale and complexity of the project as well as the countless pitfalls of
security and privacy - that continue to cause controversy - is something that
only a collective effort can handle.
Following health measures, taking care of sorroundings and the people around us
is a collective effort. Building essential digital infrastructure also needs to
be a collective project.
This is not an utopia that can only be imagined in the distant future, it
becoming reality even now. If you follow the ongoing pandemic - as you are
probably forced to - you noticed that there are many open source projects
tackling every angle that can be imagined.
The most obvious are stats and APIs for public information, but also data
scraper and analytics tools, even hardware like open source ventilators. There
are efforts to use tools like
machine learning and
more exotic new technologies as well. The list compiled by the
EU Joinup project
is massive and growing.
The virus response is one field where open-source software (and to a smaller
scale hardware) will continue to drive collective action. Open-source holds a
massive potential of innovation and efficiency, for all other fields of public
organisation, public infrastructure and services. It is time for citizens to
demand open-source solutions for essential and critical infrastructure and
services. This wont solve all - if any - problems of modern society, but it
would be a great step towards transparency and in the long run trust.
Dockertest is an integral part of the Ory Ecosystem
Ory promises to produce high quality open source software.
Dockertest is just one example of how Ory
approaches advanced software development for cloud native applications.
Ory works with a dedicated worldwide community assuring that developer friendly
security tools and innovative products in identity infrastructure are available
to everyone.
Conclusion
Thanks for taking the time to read this article! Check out
our blog for more articles and tutorials.
Should you have further questions or feedback, visit the
Ory Community Slack.
Ory Dockertest is open-source and available
on github, please consider starring the repository. It
is free and helps grow the project and community.
Sign up to
the Ory newsletter
and we will notify you of new updates to
Ory Dockertest and
other Ory projects.
