Categories
Architecture Automations Notion

Add a sequence diagram to Notion

In order to express their ideas about a project’s architecture, developers can utilise the UML sequence diagrams.

There are different ways to create and publish these diagrams, ranging from paid apps(8$/month, 100$) to free Confluence apps.

It can be a formidable task to choose from all of these options. As a programmer, a good solution could be to write the diagram in text, and later commit or convert it to an image.

PlantUML ☘️

With PlantUML, sequence diagrams can be defined in a text format, and output to .svg or other image formats via a command line tool.

Consider the diagram:

Alice and Bob sequence diagram

This can be defined in a very human-readable format inside a diagram.puml file:

@startuml
Alice -> Bob: Authentication Request
Bob --> Alice: Authentication Response

Alice -> Bob: Another authentication Request
Alice <-- Bob: Another authentication Response
@enduml

And then converted into .svg using the command line tool:

java -jar plantuml.jar -tsvg diagram.puml

There are different extensions for PlantUML. In Atlassian, one can write the definition and it will be output as an image into the document.

For the Notion workspace, there is no official integration. Instead, we can write a script and use their API.

Notion table 📝

First, the diagram should be described using either the PlantUML text format, or as a relational table.

If we want to use relations between UML objects, we need to use the latter.

Services table

In the first table, a single column with our services Alice and Bob needs to be described.

Alice and Bob services

Diagram table

In the second table, the actual diagram will be described with actors from the related services👆🏽table.

Alice and Bob diagram

Note field will be used as the action name.

Output from the tables ⚙️

To convert the tables into UML, we can use a Python script to download the Notion table, convert it to PlantUML, and output the UML image.

notion.py, “Unofficial Python 3 client for Notion API v3”, can be used to:

  • request the tables from our Notion workspace
client = NotionClient(token_v2=token)
cv = client.get_collection_view(sequence_diagram_table_url)
  • translate the tables into PlantUML
for row in cv.collection.get_rows():
    service_origin_block = row.origin[0]
    service_end_block = row.end[0]
    pum+=("\"{}\" -> \"{}\": {}\n".format(
        service_origin_block.name,
        service_end_block.name,
        row.label))
  • output the final image using plantuml.jar
subprocess.Popen(["java", "-jar", "plantuml.jar", "-tsvg", "out.puml"])

Result

After calling our script:

python notion-diagram.py https://www.notion.so/952a65008448439b9f0d82c9755a525d\?v\=0de421bd4e2b485fb624ff4edc527e0d Alice

we get our sequence diagram result

Alice and Bob diagram from Notion table

Optionally, the output could be uploaded to Notion or other services.

The script is available in tonisives repo. It is based on the official Notion blog post.

Conclusion 🖋

It is possible to create sequence diagrams in a text format, and convert them to images later.

What are the positives? 👍🏽

  • Can edit diagrams in a text format.
  • Don’t have to load a web page or a different app.
  • Can generate UML from command line, then transfer the image anywhere.

What are the negatives? 👎🏽

  • Cannot use a GUI program
  • Requires scripting and shell usage knowledge
  • More difficult diagrams can be hard to write

Related tweet:

Categories
Architecture Java Kotlin

Callback styles for async tasks

For asynchronous tasks, the actions on completion need to be handled via a callback. There are different patterns to achieve this, with each having their own benefits and shortcomings.

Interfaces

One of the oldest callback styles are interfaces or anonymous classes. They are used to great effect in Android. As an example, with okhttp library, a network request could be sent:

okHttp.newCall(request).enqueue(new Callback() {
  @Override public void onFailure(Call call, Exception e) {

  }

  @Override
  public void onResponse(Call call, Response response) {

  }
});

Interface use is very convenient, because the request and callback can be written in one line and all of the outer class properties are available in the nested function.

However, shortcomings arise when handling multiple tasks. Consider if we needed to wait for all of the requests to finish. Then, a response counting logic is required:

Response[] successfulResponses = Response[requests.size()]
final int[] responseCount = {0};

for (Request request in requests) {
  okHttp.newCall(request).enqueue(new Callback() {
    @Override
    public void onResponse(Call call, Response response) {
      successfulResponses.add(response);
      responseCount[0]++;
    }
  });
}

// wait for all of the responses

while (responseCount[1] != requests.size()) {
  Thread.sleep(1);
}

// all responses are here
println("All tasks finished:");

There is a better way to handle this kind of scenario.

CompletableFuture

Since Java8 and Android 24(or with a support lib), CompletableFuture is available. It proposes to fix the interface shortcomings like scattered callback locations, deeply nested callbacks or sequential tasks management.

With this new API, waiting for all of the answers can be done with the allOf() method:

CompletableFuture<Response>[] requests;

CompletableFuture<Void> tasks = CompletableFuture.allOf(requests);

CompletableFuture cf = tasks.thenRun(() ->
  print("All requests finished"));

// start a blocking thread to run the tasks
cf.get();

Single completions can also be observed:

for (CompletableFuture<Response> request : requests) {
  request.thenAcceptAsync(response -> {
    print("request response: " + response);
  });
}

CompletableFuture API is expansive, having different methods for creating, combining and executing tasks. Some extra benefits are:

  • Sequential task management
  • Integration with Kotlin coroutines, Streams API, RxJava

LiveData and RxJava

Recent paradigm shift in programming has been the introduction of Observable pattern. It is now even the Android’s recommended app architecture style.

What differentiates it from the previous styles, is that a single callback is used for all of the updates of a property. A common scenario is a view state, which advertises its value changes. Only the new value is advertised, irrelevant from the source of the change.

In our case it would mean that we wouldn’t get the response from the web request directly, but from an field in the ViewState object:

class ViewState {
  String response;
  String error;
}

MutableLiveData<ViewState> viewState = repository.getViewState();

// observe the view state. Observer count is unlimited
viewState.observe(this, viewState -> {
    view.setText(viewState.response)
});

// repository makes the requests internally and updates the // viewState object
repository.getNewState();

RxJava possibilities are even greater than the ones of CompletableFuture, including a rich set of chaining operators. Before jumping in, one has to consider the learning curve of a new programming paradigm.

Conclusion

There are different use cases for all of the aforementioned Async task callback styles.

Interfaces can be used for simple tasks or callbacks that are only run once and no combination is needed.

Chained tasks or more complicated process management is handled better with the CompletableFuture.

Observable pattern can be used for even greater flexibility and added benefits. It is a programming paradigm shift though, and weighing the benefits over the skill acquisition time is recommended.

Categories
Android Architecture Kotlin testing

Logging in a Java library

It can be useful to emit logs in a library. When doing so, one needs to consider when to emit, how to filter and who is responsible for printing/handling the logs. Correct logging should also be tested.

When to log

There are different reasons to emit a message, for instance on important events, undefined behaviour or different levels of debug events.

Any potentially useful message should be emitted. However, in order to not clutter the terminal, output should be refined.

Filtering the logs

A library logging level should be configurable according to user preference:

/**  Possible logging levels. */
public enum Level {
    /**  No log messages */
    OFF(0),
    /** Informational messages and errors only */
    INFO(1),
    /* Debug messages */
    DEBUG(2),
    /* All messages, including fine traces */
    ALL(3);
}

It is expected that important and error messages are emitted by default, so Level.INFO should be the default setting.

However, if finer traces are required, the filter could be set to DEBUG:

Library.loggingLevel = Level.DEBUG

The library should then filter the messages according to level:

static void logDebug(String message) {
    if (loggingLevel >= Level.DEBUG) {
        loggingFramework.logDebug(message);
    }
}

Example of filtering implementation in hmkit-android library.

Choosing a logging framework

The user could be using any java subsystem, or maybe emitting the messages to a web service. For this reason, the library should never output to System.out.println or android.util.Log. Instead, it should be an interface from where the logs are emitted, and it should be up to the user to choose where the messages are output in the end.

A popular logging facade is slf4j. api part of it should be included in our library:

implementation 'org.slf4j:slf4j-api:1.7.25'

Then logs can be emitted through a Logger instance:

// sample informational log
logger = LoggerFactory.getLogger(Library.class);
...
logger.info("Library initialised")

Our library user could then continue using his favourite logging framework and add an slf4j binding to see the messages. For instance, a Timber binding:

implementation 'at.favre.lib:slf4j-timber:1.0.0'

Testing the emitted logs

To verify the emitted logs, a test setup is necessary. Mockk can be used to mock the slf4j and verify calls to it’s Logger.

For this to work, each test could inherit from BaseTest, which initialises the mock:

lateinit var mockLogger:org.slf4j.Logger

@BeforeEach
fun before() {
  mockLogger = mockk()
  mockkStatic(MyLoggerFactory::class)
  every { MyLoggerFactory.getLogger() } returns mockLogger

  every { mockLogger.warn(allAny()) } just Runs
  every { mockLogger.debug(allAny()) } just Runs
  every { mockLogger.error(allAny()) } just Runs
}

This class could also contain convenience lambda methods to assert the emitted logs:

fun debugLogExpected(runnable: Runnable, count: Int = 1) {
    runnable.run()
    verify(exactly = count) { mockLogger.debug(allAny()) }
}

From the derived class’s test method, an assertion can be then written about the log message:

@Test fun invalidStartControlControlModeThrows() {
    debugLogExpected { 
        val action = Library.resolve()
    }
}

This test will succeed if one debug message is emitted to the slf4j interface.

Please see auto-api-java as an example of using this pattern.

Conclusion

Logging in a library can be very beneficial. The maintainer should however be aware of the library user’s perspective, and filter too precise logs by default. Log printing should also be left to the user, and correct log emittance should be tested with unit tests.

Categories
Android Apps Architecture testing

Part 2: Testing with MockK and Koin

One of the best things about MVVM is the use of separation of concerns principle which by design enables testing of each component in isolation. View, ViewModel and Model are all separated and thus easily testable.

When thinking of testing, then unit testing comes to mind first and for that mocking of dependencies is required.

Mockito vs MockK

After investigating Google’s demo project, it seemed Mockito was the way to go with mocking and verifying tests.

What I soon realised, was that it wasn’t the most convenient library to use in Kotlin, and I also had problems with just getting it to work. I then discovered MockK, written in Kotlin, which was easy to setup and thus made a perfect choice for my project.

Consider mocking a network response:

Mockito

val call = successCall(contributors)
`when`(service.getContributors()).thenReturn(call)

MockK

val call = successCall(contributors)
every { service.getContributors()} returns call

I choose MockK’s every / lambda style over Mockito’s `when`.

The only thing missing from MockK is verifying constructor calls, for which there is a Github issue. Because of this I needed to refactor my code and inject the dependencies, instead of constructing them.

MockK setup

Separate libraries are required for unit and instrumentation tests:

testImplementation "io.mockk:mockk:$version"
androidTestImplementation "io.mockk:mockk-android:$version"

This is enough to start mocking dependencies in Unit tests. For instance, mocking a network client:

val client = mockk<RepoClient>()

For instrumentation tests, you need to launch the real activity and thus need a separate Test App class and mocked Koin modules. Read about this in Instrumentation section below👇🏽👇🏽

Testing the Repository

All the classes can be covered with unit tests. In the @Before block, you should create the class with mocked dependencies:

@Before
fun before() {
    client = mockk<RepoClient>()
    repository = RepoRepository(client, /*more mocks..*/)
}

Then, in your test, you can mock answers from your dependencies and verify expected Repository behavior.

// mock the repository observer
val observer = mockk<Observer<Resource<List<Repo>>>>()
// call getRepos() and observe the response
repository.getRepos().observeForever(observer)
// verify repos are fetched from network
verify { client.getRepos() }
// simulate that network data was stored to db
dbData.postValue(repos)
// verify getRepos() observer was called
verify { observer.onChanged(Resource.success(repos)) }

With this style you can write unit tests for all of your classes.

UI Instrumentation tests

Instrumentation tests are used to verify what is visible to the user. The app will launch with mocked ViewModel and the tests can then verify the UI state.

Setup

For instrumentation tests, you have to set up a custom Test App and its companion Test Runner which is then used to run the tests. Needless to say the setup is pretty complicated but the tests are worth it after the initial hurdle.

@Before and @After

Before the UI test, the ViewModel should be mocked and its responses set. Then the tested activity/fragment should be launched.

@Before
fun before() {
    // mock the ViewModel
    loginRequest = MutableLiveData()
    loginViewModel = mockk(relaxed = true)
    every { loginViewModel.user } returns loginRequest

    module = module(true, true) {
        single { loginViewModel }
        // mock other dependencies
        single { mockk<MainViewModel>(relaxed = true) }
        // ... etc
    }

    loadKoinModules(module)
    // launch the activity
    scenario = launchActivity()
}

After the test the activity should be closed and Koin modules unloaded so the next test can start with cleared objects.

@After
fun after() {
    scenario.close()
    unloadKoinModules(module)
}

The test

Then, as in Unit tests, you can mock updates from ViewModel and verify expected View behavior. You can also simulate input from the view.

For instance, verifying a Toast message after invalid login:

// input wrong credentials
inputCredentials("wrong", "wrong")

// click the login button
onView(withId(R.id.loginButton)).perform(click())

// verify ViewModel's login is called
verify { loginViewModel.login(any(), any()) }

// simulate error response
loginRequest.postValue(Resource.error(getString(R.string.invalid_credentials), null))

// assert error toast shown
onView(withText(R.string.invalid_credentials))...

Similarly to this, all of the views can be tested.

Conclusion

Although setup and complexity of Android tests could be improved, it is essential for delivering a quality application.

I can say from my experience that numerous times seemingly irrelevant tests have failed after writing new code. These failed tests, if not caught, would have meant bugs in production.

Reference

Please have a look at the unit and instrumentation tests in the sample project’s source code.

Categories
Android Apps Architecture

Android App Architecture, Part 1

Not being familiar with modern Android App architecture and coming from the ViewController world, it is confusing jumping into the recommended MVVM architecture. There are also some parts of the official guide that are left for the reader to figure out, so I will write about my experience with implementing it in my repo browser project.

Choosing dependencies

Besides native Architecture dependencies, there are some parts of the setup that are not defined in the guide. Two of the main ones are the networking and dependency injection libraries.

Networking

The guide uses Retrofit for networking. For me, it made sense to use Volley. I like that I’m in control of the requests that I write, and am not dependent on the Retrofit abstraction of mapping requests to data objects. I can be sure that if there will be customisation required for my requests, Volley can handle it.

Dependency injection

There are many DI libraries, with the most popular one being Dagger. For me, it seemed complicated to get started with and with a lot of boilerplate. I didn’t see any drawbacks from using Koin, so I went for that one instead.

Here are the final dependencies for my project.

Koin setup

After planning and creating the required Activity, ViewModel, Networking, Database and Repository objects, it was time to set up the Koin modules. For my use case, I created singletons and viewModels:

// repository Database
single { get<AppDatabase>().repoDao() }
// repository networking
single { RepoClient(context, get()) }
// repository repository to merge data from database and network
single { RepoRepository(get(), get(), get()) }
// the viewModel
viewModel { (handle: SavedStateHandle) -> 
    RepoListViewModel(handle, get(), get()) } 

Notice the viewModel with SavedStateHandle argument. This allows access to the saved state and arguments of the associated Activity.

Other singletons required were: Thread executors, Room database, SharedPreferences

Here is the Koin setup.

Repository setup

For fetching the repositories from Github and storing/reading them from the database, 3 components were required:

  • Network client
  • Database object
  • MediatorLiveData to merge network/db data

After that, accessing the data in LiveData format is straightforward:

val repoResource = object : NetworkBoundResource<List<Repo>, List<Repo>>(executor) {
    override fun saveCallResult(item: List<Repo>) = repoDao.save(item)

    override fun shouldFetch(data: List<Repo>?): Boolean {
        return data == null || data.isEmpty() || rateLimit.shouldFetch("repos")
    }

    override fun loadFromDb() = repoDao.getRepos()

    override fun createCall() = repoClient.getRepos()

    override fun onFetchFailed() = rateLimit.reset("repos")

}.asLiveData()

Check out the repository source code.

Populating the view

For the last part, the view is updated according to the LiveData<List<Repo>> result. If there is data, the ListView is populated with a DataBinding adapter:

viewModel.repos.observe(viewLifecycleOwner) {
    // update UI
    it.status.showWhenLoading(listProgressBar)

    when (it.status) {
        Status.SUCCESS -> {
            adapter.submitList(it.data)
        }
        Status.ERROR -> {
         Toast.makeText(this@RepoListFragment.activity,
             it.message, 
             Toast.LENGTH_LONG).show()
        }
    }
}

Conclusion

These are the main steps of creating an Android MVVM skeleton app. Of course there are more details(networking, database), which can be discovered in the GitHub repository.

Stay tuned for Part 2, where I will write about testing the app with Mockk.

Reference

Project code in GitHub.

Categories
Android Architecture NDK

Sharing native code between Android and Java projects

In High-Mobility, we have separate libraries for Android and Linux. For each of these we use native code that handles transport protocol. After implementing JNI for both of the platforms, we realized it would make sense to use a shared JNI module instead.

Android setup

At first we started developing for Android, and went for the quick solution of including C submodules and setting up the JNI classes/Makefile.

We ended up with structure:

This structure already had the problem that the C developer had to work in the Android project to update the native code. This means he had to follow the Android project’s branches, and not develop in the C repository independently.

Linux setup

Later we also needed a Linux library. Following the success of the Android project, we created a project with similar structure:

Now our C developer had even more problems. He had to follow the branches of both Android and Linux projects, and update the native module’s branches on both. Since our JNI code is the same, he also had to copy/duplicate the JNI code from one project to the other.

Solution: Shared JNI module

It was clear that restructuring of the projects was necessary. We would need to create a new package that includes the shared items: C submodules and JNI classes. Makefiles are different for Linux and Android, so these would be retained in original projects. 

This is the final project hierarchy:

Now, when our C developer adds functions to the JNI code, he only has to update the shared “Core JNI” package. Our Java projects can then update to the new “Core JNI” branch.

Reference

HMKit Android: https://github.com/highmobility/hmkit-android

HMKit Linux: https://github.com/highmobility/hm-java-oem

HMKit Core JNI: https://github.com/highmobility/hmkit-core-jni