Our team was too curious to compare Flutter and native Android apps, so we decided to create two application projects; one with Native Android and the other with Flutter, with similar UI, RESTful APIs, and processes. Doing so allowed us to compare the performance of both apps side by side.
Before we start showing the results of the various tests we performed on these applications, let us give you a detailed context of what we did. Here is a quick overview of the application details, architecture, and testing aids our team utilized in the process
We developed a weather app that displays a list of weather details for the user’s current location and previously searched locations. For the weather API, we used Open Weather’s free tier APIs. The user can look for weather details for a given location in the search screen by tapping on the map or by searching with the city name. We used Geocoder for location auto-complete and Open Street Map for the map view.
As we wanted to test the performance of Android and Flutter applications, we kept our apps relatively small, with only three screens: Home, Search, and Details.
At Aubergine Solutions, we believe in following a standardized process to ensure that your applications are of high quality and foolproof. Driven by this habit, we followed the same procedure in this case. Applications developed with each platform use the recommended guidelines of the platform developers to get the best performance. We used BLoC State Management for developing the Flutter application and MVVM + Coroutines with Single Activity Architecture for Android application development.
For Flutter App, we used Integration Testing to measure the performance. We considered the metrics provided by the integration_test library during the performance profiling. For Android App, we performed Macrobenchmark and Microbenchmark to measure the app performance.
Both applications were benchmarked on the same device, the Google Pixel 6a, with no background tasks running.
Now to the benchmark results!
We tested our Android Native and Flutter applications against a certain set of benchmarks. It was essential to do so, as selecting benchmarks helps us measure the efficiency of both apps in the shortest amount of time. Let’s see which benchmarks were picked, and how both technologies performed.
We measured both applications, Android and Flutter, with Macrobenchmark to ensure consistent results. As you can see in the results, Android Native was much faster than the Flutter app in terms of app startup time.
Results
+---------------+---------+---------+ | App Startup Benchmark | +---------------+---------+---------+ | Type | Android | Flutter | +---------------+---------+---------+ | Minimum | 207ms | 336ms | | Median | 218ms | 360ms | | Std.deviation | 6.34ms | 25.84ms | +---------------+---------+---------+ *Lower is better
So in this test, Android outperformed Flutter in the benchmarks.
When an end-user lands on an app page, their first experience is related to how quickly the page/screen loads. This is a clear difference-maker in understanding how both apps are performing.
We measured how long it took for a screen to load during navigation.
Before diving into these benchmarks, we must understand what the results mean when they are output.
Percentiles
This is defined as PXX where XX is the percentile number.
Few examples:
‘P50 = 10.5ms’ means 50% of all frames should be faster than 10.5 ms.
‘P99 = 25ms’ means 99% of all frames should be faster than 25ms.
Also, in the graphs, Build(P90) means each of 90% of frames took less than this much amount of time to be produced by the CPU. Render(P90) means how fast 90% of frames were Rasterized on the screen. The same goes for Build(P99) and Render(P99).
Rasterisation is the technique of transforming a frame described mathematically in vectors into pixels to be displayed on the screen.
The ideal time for Rasterization can be found with this simple equation
1000 / (fps of the display) = Ideal Rasterisation time (ms)
Anything greater than this number may result in dropped frames or visible jank/stutter. The further it is from the ideal value, the more jank it may cause.
i.e. for our testing device’s 60fps display, the ideal rasterization time is 16.66 ms
For Flutter:
We gathered detailed information about the page navigation using the watchPerformance() method, which provides detailed insights about UI and raster threads. You can learn more about these metrics here. To measure the time taken to complete the transition and open the new page, we used a StopWatch.
For Android:
We used Macrobenchmarks for Android to measure the loading times of the screen. We conducted Android benchmarking five times, providing results for each run as well as the minimum, maximum, and median values.
Navigating to Search Screen by clicking the search icon.
Surprisingly, the majority of frames in Flutter were produced faster, but some took significantly more time than Native Android. Even though Flutter built frames faster, it fell behind Native Android when it came to displaying the frame on screen. See the Render(PXX) columns.
+-------------+---------+---------+ | Home to Search Benchmark | +-------------+---------+---------+ | Type | Android | Flutter | +-------------+---------+---------+ | Build(P90) | 7.8ms | 4.0ms | | Build(P99) | 12.3ms | 29.8ms | | Render(P90) | 9.6ms | 25.9ms | | Render(P99) | 22.8ms | 55.1ms | +-------------+---------+---------+ *Lower is better
As explained above, the ideal time for a frame to be rasterized for a 60fps display is 16.66 ms, and we can see that more than 90% of the frames with the Android counterpart were ready to be displayed on the screen within that time.
Navigating to Details Screen by tapping on a list item.
Again, Flutter benchmarking was far ahead in computing the frames, but when it comes to being consistent and rasterizing the frames, Native Android takes the lead.
+-------------+---------+---------+
| Home to Details Benchmark |
+-------------+---------+---------+
| Type | Android | Flutter |
+-------------+---------+---------+
| Build(P90) | 15.9ms | 4.0ms |
| Build(P99) | 18.3ms | 25.6ms |
| Render(P90) | 11.5ms | 11.3ms |
| Render(P99) | 15.5ms | 53.8ms |
+-------------+---------+---------+
*Lower is better
To measure map responsiveness, we performed “fling” operations to the bottom, right, and top twice. Surprisingly, Flutter outperformed Native Android in all cases! Could this be because of only having to refresh the map part of the screen instead of rendering entire page like in Android?
+-------------+---------+---------+
| Map Responsiveness Benchmark |
+-------------+---------+---------+
| Type | Android | Flutter |
+-------------+---------+---------+
| Build(p90) | 11.3ms | 4.0ms |
| Build(p99) | 18.6ms | 7.9ms |
| Render(p90) | 14.1ms | 5.5ms |
| Render(p99) | 20.2ms | 7.8ms |
+-------------+---------+---------+
* Lower is better
We performed a CPU-intensive operation that loops many times and performs a math function to compute a result. Here is the Kotlin code for the function for reference.
Results
As expected Android Native outperformed Dart computation by calculating results with 50% less time.
+-----------+---------+---------+ | x | Android | Flutter | +-----------+---------+---------+ | Avg. Time | 13.4ms | 26.4ms | +-----------+---------+---------+ * Lower is better Find out how we created an Android app for scaring pigeons using ML Kit.
With native Android development, it does not come with many preloaded libraries in the APK, resulting in a much smaller APK size compared to Flutter. Both APKs were created with the minifyEnabled flag set to true. The native Android APK was drastically smaller than the one generated for the Flutter app; 77.8% smaller, to be specific.
When using Flutter, the overall development time, especially the UI development time, is generally less when compared to native platforms. Additionally, there are tools and packages available which allow us to save our precious time from redundant and mundane tasks, such as automation tools for generating and parsing data models, and startup app providers (like very_good_cli) for kick-starting new projects with all the necessary features pre-loaded.
Learn how to create a note-taking app using Flutter/Dart.
Similar to Flutter, the overall time for native Android app development is generally not much, but with Flutter, you can have your app built for two platforms simultaneously. Android has good first-party support with Jetpack Libraries.
Undoubtedly, Flutter has good community support, but being a mature framework, Android has far better support for debugging and resolving issues. Additionally, you will not find yourself stuck on some major issue that exists at a platform level itself.
As for the development time for the weather app, each screen was developed in around 4–8 hours, including the map view. The entire project, including the performance testing, took around two weeks; however, the majority of the time was spent on the performance testing for both apps.
Many teams or entrepreneurs get confused by questions such as, “What is better: Native development or Flutter?” or “What should I develop my app with: Android native app development or a Flutter-based solution?“
Well, it depends on your requirements. If your application needs to utilize the device hardware, do a lot of heavy work or perform long-running tasks on the device, or compute data with complex algorithms, it is always a good idea to choose a native application so that you can squeeze every last bit of the performance from the device hardware.
But Flutter is indeed production ready and has showcased an excellent app portfolio. If your product app is simple and talks to a server via web services, shows data from the local database, or provides soft utility to the users with occasional intensive tasks, you can go with the cross-platform application as it would not have any noticeable difference in performance for the end user. Instead, Flutter app development will allow you to easily include both Android and iOS within a single development project, resulting in a smaller team size and saving a lot in development and management costs.
Both Flutter and Android native applications are great, depending on your requirements, industry, and target audience. If you are unsure which technology is best for you, let our software product development specialists make a data-driven decision for you. Start a free consultation now.
