Auto Added by WPeMatico

Realm Kotlin Multiplatform SDK

Nabil Hachica, Android Engineer, from MongoDB Realm shares their story of creating a pure Kotlin-first SDK for use by developers.

Realm is an open-source, easy-to-use local database that helps mobile developers to build better apps faster. It launched 8 years ago with the release of the Realm Java SDK, and since then we’ve worked to make sure our API is always improving to reflect what mobile developers need.

As Kotlin has grown to become the recommended language for building Android apps, we’ve made iterative improvements to our Realm Java SDK to make sure the API is compatible. But even with various releases over the past few years — via providing helper extension functions to improve using the Realm API with Kotlin — we knew Realm would ultimately benefit from the design and implementation of a pure Kotlin-first SDK for use by developers.

In the post, we take you through what motivated the Realm Engineering team’s decision to build a new SDK from the ground-up, and we share some of the key design principles that guided the development of Realm’s Kotlin Multiplatform SDK.

Background

As usage of Kotlin grew among Android developers, Realm’s engineering team started to use Kotlin in our testing environment. This led to introducing helper functions like Query with generic type; executeTransactionAwait – which runs transactions inside coroutines — and JSR305 annotation to improve the strict null-safety type system with Kotlin.

But even with this work, there were still architectural shifts that were difficult to solve, like users adopting unidirectional dataflows based on immutability. The wider Realm community also asked us to provide a more idiomatic Kotlin API that would enable a type safe query builder, among other API improvements. We knew some of these enhancements would require a fundamental refactoring and we wanted to meet our users’ needs. So Realm’s Engineering team started to design and implement a pure Kotlin first SDK for the mobile ecosystem.

Why a Multiplatform Library?

At the same time we began to lay the foundation of our Kotlin SDK, Kotlin Multiplatform started to mature, and we quickly realized we should build Realm’s new Kotlin SDK as a Multiplatform library. There were surprisingly few multiplatform libraries for the persistence layer, and as a multiplatform mobile database, we knew Realm could help fill this gap for developers. We had an opportunity to allow Kotlin developers to build cross platform applications in the same way that Realm’s developers use our React Native and Xamarin SDKs today.

Challenges

Porting the existing Realm-Java code to work on Multiplatform was an interesting (and challenging) task for Realm’s engineers because we currently use some technologies that are platform dependent.

To define a Realm schema, we use the annotation processor to generate a proxy class and then use the Gradle Transform API to do bytecode weaving with Javassist. Both tools are Kotlin JVM specific and are not compatible with Kotlin/Native.

Similarly, Reflection is only partially supported on Kotlin/Native, so any API design based on dynamic runtime instantiation would simply not work — like the Class.forName or class.newInstance primitives.

And, concurrency and the memory model are fundamentally different between platforms. The Kotlin/JVM has a Garbage Collector, whereas Kotlin/Native relies on reference counting with a cyclic collector.

To successfully build a multiplatform Kotlin Native SDK for Realm, we’d need to figure out how to re-architect the SDK in a way that would address these challenges.

The Solution


Note from the Kotlin team: Compiler plugins and compiler internal API are not stable yet and will likely change in the future.


Compiler Plugin to the Rescue

Realm’s engineering team quickly decided we would need to build our own fully-fledged compiler plug-in because existing options posed too many constraints.

The Kotlin Symbol Processing API is limited to generating new Kotlin code instead of modifying existing classes, meaning it wouldn’t suit our needs. Realm had used the Kotlin Annotation processor (KAPT) in our Kotlin support for the Realm Java SDK, but it didn’t have a native annotation processing system. Instead, the Realm Java SDK generates Java stubs containing all the declarations and annotations without the methods body. It then runs the Java annotation processor on those stubs, which feeds back to KAPT and Kotlin compiler for another round. This makes compilation and refactoring slow, and isn’t available for Kotlin/Native.

At a high level, the Realm compiler plugin‘s main goal is to make defining Realm models as simple as constructing a regular Kotlin class. This is achieved by modifying the Intermediate Representation (IR) of the compiled Kotlin code in order to change the behaviour of the property accessor (getters and setters).  Once this is done a developer can now call Realm’s runtime APIs so that read/writing changes the value of the property instead of the backing field. This is done using a Lowering pass on the IR.

The other task of the compiler plugin is to collect all Realm model classes in order to automatically create a schema definition of a Realm. Since the Compiler plugin API was not yet stable and documented (1.4.30 makes JVM IR Beta stable), we reverse engineered the IR code we needed to write in order to obtain the desired generated code.

Here’s what that looks like:

First, we write the Kotlin code we want the compiler to generate in a classic Kotlin/JVM project. Then we add the compiler flag to use IR and to dump the content of the IR.

This will dump a text file containing the IR code generated by the compiler before transforming it into JVM ByteCode.  We then use this IR dump to work out what IR API calls we need to use to produce the desired output. We also use kotlin-compile-testing library to unit test the IR output from the Realm compiler plugin that matches the expected IR.

Releasing Native Resources

Under the hood the Realm SDKs use a C++ storage engine. The compiler plugin adds infrastructure to link the Kotlin objects with the underlying C objects by adding a synthetic native pointer property. This holds the value of the C/ C++ pointer, allowing access to the underlying persisted value inside the storage engine.

In order to avoid leaking memory, the native pointer needs to be closed when the original Kotlin object goes out of scope.
Since Kotlin/JVM and Kotlin/Native have different memory models, we decided to use two approaches. For Kotlin/JVM we rely on the same technique we use in the current Realm-Java SDK, which is based on the usage of the reference API to track pointers that need to be cleaned once their original Java counterpart is eligible for GC.




For more information about reference API and phantom reference check out this excellent talk by Bob Lee

Kotlin/Native doesn’t have a Garbage collector. It uses reference counting with a cycle collector instead. Recently the Kotlin/Native team introduced the Cleaner API (see https://github.com/JetBrains/kotlin-native/pull/4362). This API can be used similarly to a Finalizer/Destructor block that runs after the encompassing object has been disposed of. We use this API to perform cleanups and free native pointers.

Encapsulating the native pointers in an object with the above deallocation strategies, which hook into the normal memory management, frees us from handling explicit deallocations where we use native pointers.

Native code abstraction

Accessing the custom C++ storage engine used by the Realm SDKs varies between Kotlin/JVM and Kotlin/Native. To simplify this, we added a C layer on top, which makes it easy to use tools like cinterop to interface and generate Kotlin stubs. The C layer also allowed us to avoid writing manual JNI code to access the C++ layer, thanks to SWIG tool that generates the corresponding Java and JNI stubs.

We also leveraged the expect/actual mechanism from Kotlin Multiplatform to build a module that hides the specifics of accessing Cinterop and SWIG layers in a platform agnostic manner. This allowed us to focus the design of the library API in a platform-independent way, and to do it in another module that will become the user facing public API.

This process encourages to think about features — like querying and writing to the database — in a way that’s not platform-specific.

Any platform dependent implementation details — like cleaning up native resources — do use different fundamentals per-platform (Garbage Collector for Kotlin/JVM and the Cleaner API for Kotlin/Native). This is the type of feature that should be kept inside the Native Module Abstraction layer and not be implemented in the public library module.

By isolating these low level constructs in a separate Multiplatform module, we managed to scale the development of the new Kotlin SDK within the Realm team. All the low level building blocks are isolated behind an internal platform agnostic API. This also opens up API contributions from the community without worrying about the inherent complexity of C/C++, Kotlin/Native, or JNI.

Conclusion

Now, we’re proud to say that Realm’s engineering team managed to bootstrap the new Realm Kotlin SDK as a Multiplatform library. We believe that the infrastructure is in place to encourage other authors to build Multiplatform libraries, too.

Of course, as an alpha release, we’re still polishing elements of the solution. Realm engineering has been working in lockstep with the team at JetBrains, and we have full confidence that improvements will land soon.
As we keep working, we’d love your feedback. Try the new Realm Kotlin SDK out at https://github.com/realm/realm-kotlin, and join our discussion around the design of upcoming features here. We want your opinions and suggestions as we keep building.

Author: Nabil Hachica, Android Engineer, MongoDB Realm

Continue Reading Realm Kotlin Multiplatform SDK

Kotlin Multiplatform Mobile goes Alpha

Kotlin Multiplatform Mobile (KMM) is an SDK that allows you to use the same business logic code in both iOS and Android applications. Today KMM goes Alpha, and you can start sharing business logic in your mobile apps with it right away. It includes the new KMM Plugin for Android Studio, which allows you to write, run, test, and debug shared code in the same IDE. Join such great teams as VMWare, Autodesk and Yandex who are sharing code in their mobile apps using Kotlin. It has never been so easy!

file

What is Kotlin Multiplatform Mobile?

Kotlin Multiplatform Mobile (KMM) is an SDK for cross-platform mobile development provided by JetBrains. It uses the multiplatform capabilities of Kotlin and includes various tools and features designed to make the end-to-end experience of building mobile cross-platform applications as enjoyable and efficient as possible.

The Android and iOS versions of an application often have a lot in common, but they can also differ significantly – especially in terms of their UI – from subtle variations in scrolling behavior to completely divergent navigation logic. At the same time, the application’s business logic, including such features as data management, analytics, and authentication, is often identical. That’s why it’s natural to share some parts of an application across platforms while keeping other parts completely separate.

With KMM, you can get this flexibility and retain the benefits of native programming. Use a single codebase for the business logic of iOS and Android apps and write platform-specific code only where it’s necessary, to implement a native UI or when working with platform-specific APIs.

file

KMM is seamlessly integrated with your mobile project. Shared code, written in Kotlin, is compiled to JVM bytecode with Kotlin/JVM and to native binaries with Kotlin/Native, so you can use your KMM business-logic modules just like any other regular mobile library.

Sharing Kotlin code between mobile platforms has already saved a lot of time and effort for many companies. Here are some inspiring stories:

  • Quizlet migrated their business logic from a shared JavaScript approach to KMM and drastically improved the performance of both their Android and iOS applications.
  • Fastwork introduced KMM for the domain and data layers of their application to share business logic and API service between mobile platforms, significantly boosting their team’s productivity.
  • Yandex.Disk started out by experimenting with the integration of a small feature, and when the experiment proved successful, they implemented their whole data synchronization logic in KMM.

The remarkable results achieved by these and many over teams have been a big part of our inspiration, and we hope that their stories will encourage you to start sharing code between mobile platforms with KMM.

Enjoy creating mobile apps with KMM

KMM is still quite new to the mobile development world, but we believe that it is a true game-changer. That’s why we allocated a separate team inside Kotlin to push KMM forward and provide the best developer experience for the creation of cross-platform mobile applications with Kotlin.

As a first step toward this, today we are happy to present to you the first preview of the KMM plugin for Android Studio along with the new KMM developer portal.

Let’s start our KMM Journey!

Work in a familiar environment

With the new KMM plugin, you can write, run, test, and debug shared code in Android Studio, without switching to other IDEs. To install it, select Preferences | Plugins, search for the Kotlin Multiplatform Mobile plugin in Marketplace, and press the Install button.

Configure a new application in a few clicks

With the New Kotlin Multiplatform Mobile Project wizard, you can create a ready-to-run mobile application project with a shared codebase in just a few clicks. To see how it works on both platforms, just select the run configuration you need and click the Run button.

file

The generated project will contain examples of how to connect to platform-specific API and basic test coverage. You can use this wizard to explore the basic features of KMM or as a scaffolding tool for your new KMM production application.

Follow this tutorial to install the plugin and create your first multiplatform application with the wizard.

Introduce KMM to your existing project in one step

You also can start your KMM journey right in your current project: the shared Kotlin code can be easily integrated into your existing code just like any other regular dependency. The New Kotlin Multiplatform Mobile Module wizard will help you add a new KMM module to your project, so you can use the code you’ve already written in Kotlin and refactor it to make it compatible with iOS.

Check out this guide to integrating KMM into existing projects to learn how to design the architecture of multiplatform projects and organize the migration to it.

Work with both platforms within one IDE

You no longer need to switch IDEs and have Xcode running to check that your code works correctly on both mobile platforms. With the new plugin, you get integration with your iOS device and simulator directly in Android Studio. This gives you the following capabilities:

  • Run and debug the iOS application on the simulator or a device with the predefined run configuration.
  • Run and debug common and iOS specific tests on the iOS simulator by simply clicking the gutter icon on a test class or method.

For both the application and tests you can set breakpoints and debug your code on iOS.

Integrate with the iOS ecosystem without any overhead

As mentioned above, KMM is seamlessly integrated with your mobile project. For Android, you can use Kotlin for every part of your project. At the same time, KMM provides tight integration with the iOS development process, because of Kotlin/Native’s interoperability with Objective-C/Swift, the integration with the CocoaPods dependency manager, and the ability to use platform-specific APIs with the expect/actual pattern.

Enjoy working with Kotlin from Swift code with Kotlin/Native

Kotlin/Native provides bidirectional interoperability with Objective-C/Swift. Kotlin modules can be fully used in Swift/Objective-C. You can use Objective-C frameworks and libraries in Kotlin code, as well as Swift libraries if their API is exported to Objective-C with

@objc

.

Learn more about Kotlin/Native’s interoperability with Swift/Objective-C.

Easily manage iOS dependencies

Kotlin/Native provides integration with the CocoaPods dependency manager. You can add dependencies on Pod libraries stored in the CocoaPods repository or locally, and you can also use multiplatform projects with native targets as a CocoaPods dependency (Kotlin Pod). Manage Pod dependencies directly in Android Studio and enjoy full coding support, including highlighting and completion.

This is how the typical KMM library configuration looks:

kotlin {
	android()
     ios()
    
     cocoapods {
         summary = "CocoaPods test library"
         homepage = "https://github.com/JetBrains/kotlin"
         pod("AFNetworking", "~> 4.0.0")
         podfile = project.file("../ios-app/Podfile")
     }
 }

Connect the library to your iOS project as a Kotlin Pod by adding its name and path to the

Podfile

:

 use_frameworks!

 target 'ios-app' do
         pod 'kotlin_library', :path => '../kotlin-library'
 end

Learn more about CocoaPods integration.

Learn how to create better multiplatform mobile apps on the new Developer Portal

The new KMM Developer Portal is designed to make your journey more comfortable and productive, from your very first experience with KMM all the way through to using it in production.

The documentation section is your trusted resource when you are working with KMM. You will find content for all levels of expertise, from tutorials to get you started to more advanced development topics. It contains useful information about common tasks, like organizing networking and data storage layers and even organizing the development process around KMM.

Pick up new ideas about the benefits of using KMM in your application by learning from the first-hand experience of a variety of teams who are already using KMM in their products in the case studies section.

The power of the community

We are indebted to the following companies and independent developers whose investment of time, knowledge, and expertise were crucial for this release: Kevin Galligan, Alec Strong, Arkadii Ivanov, Ben Asher, John O’Reilly, Louis CAD, Kris Wong, Aleksey Mikhailov and and many other incredible engineers. Thank you for reporting issues on YouTrack, participating in product research and interviews, sharing stories about your experience using KMM, and creating incredible multiplatform libraries!

We also want to thank the teams who helped us create content for the new Developer Portal: Touchlab, IceRock, and Kodein Koders. Your work will help thousands of developers delve deeper into Kotlin Multiplatform Mobile and harness its full power.

You rock, folks!

Try it now

With all these cool features, we are happy to announce that KMM is now in Alpha. This means that the Kotlin team is fully committed to improving this technology and helping it evolve. It is also a signal that the product will develop quickly. We’ll listen to your feedback and provide fixes and improvements as soon as possible. Please help us increase the stability of KMM by reporting any issues you find via our tracker.

Going Alpha means that everything is ready for you to start using KMM to share business logic code between mobile platforms. You can create a new project or easily integrate a KMM module into existing ones. In both scenarios, the work can be done with minimal cost and you can always go back to fully native development. The community is growing fast, and you can influence the development of the whole ecosystem by participating in it.

So now is a great time to start using KMM!

To get the best experience of taking a deep dive into KMM and to discuss all the features you plan to explore, join the community of multiplatform enthusiasts in Slack (get invite here) and subscribe to the “kotlin-multiplatform” tag on Stack Overflow.

Though we try our best, we can’t guarantee a smooth migration just yet. However, we’ll try to avoid compatibility issues as much as possible by using feature flags and providing migration guides for new versions.

Share your feedback

This release is just the beginning of the journey, but we need your help along the way. We look forward to all your feedback, both bad and good. Become an inspiration for others by sharing your stories of using Kotlin Multiplatform Mobile, and please share any ideas you have about how we can improve KMM.

Let’s Kotlin!

Continue Reading Kotlin Multiplatform Mobile goes Alpha

End of content

No more pages to load