What is coming for developers at Google I/O 2024 — Gemini in IDEs and Chrome, Firebase SQL Database, Genkit, Project IDX, Kotlin Multiplatform, Checks, Agents
8 min readMay 17, 2024
First of all, we are so sorry for situation in Rafah, Gaza, Palestine. We call for an immediate ceasefire and urge the allowance of humanitarian aid. Let Palestine be free from genocide.
Gemini API & Google AI Studio
- Gemini 1.5 Flash is now available on Google AI Studio in over 200 countries and territories, offering low latency.
- Gemini’s long context capabilities allow it to handle large materials, such as Figma designs, and generate front-end code from these designs (similar to Locofy).
- You can add various materials (system instructions, blog posts, voice memos) to Google AI Studio. Then add new materials in your prompt a to receive generated responses from Gemini, such as creating a blog post.
- You can integrate the Gemini API by AI Studio to directly in your app.
Context Cache
When your prompts include regular data (like referencing materials), the context cache can be used, reducing costs. These materials can include papers, legal documents, school assignments, and employee training materials.
Gemini Nano
- Generative AI Model on-device:
- Runs directly on the device with low latency responses, protecting data and privacy. Enables features like suggested replies in messaging apps.
- Also it works when no cellular network
- It will be at core of Android. Gemini Nano and AICore on Pixel 8 Pro and Samsung Galaxy S24 series, with more devices to follow.
- Also you can create on-device ML models and pipelines by using MediaPipe
Official Support for Kotlin Multiplatform 🚀
- Kotlin Multiplatform support will be continue to add Jetpack Libraries
- Google Docs app migrating to KMP to share business logic across Android, iOS, and Web. Future migrations planned for Gmail, Calendar, Drive, Meet, etc.
- KMP will enhance productivity by sharing business logic across platforms.
Android
- Compose features and performance improvements are continued such shared element transitions.
- To facilitate better responsiveness across different screen sizes, new layouts have been implemented
- AI-powered stylus handwriting recognition transforms handwriting into text in both Views and Compose.
Widget glance, resizable emulator, firebase devices
Gemini in Android Studio
- Gemini will analyze Crashlytics crashes and suggest solutions.
- Allows code transformation, documentation, commenting etc. You can add new prompts to result of previous prompt by Refine button.
- Examples of transformations include optimizing code, documenting code, translating strings, and writing tests.
🥳 😎 You can build Jetpack Compose screens from given designs using Gemini 1.5 Pro’s multi-modal and long context features.
- Multi-modal features will be available later this year, while transformations and refinement are available in the Koala version of Android Studio.
Project Gameface
- Project Gameface has launched for Android, allows users you to control a novel cursor on their Android devices using their head movements and facial expressions. It is open-sourced.
Web
Gemini Nano in Chrome
- Starting in Chrome 126, Gemini will be built into the Chrome Desktop client, enabling the use of on-device AI.
- With Help Me Write, users can fill short-form content like product reviews, social media posts, and customer feedback forms, backed by on-device AI. Google has fine-tuned Gemini and optimized it for Chrome for that.
- 🚀As a developer, you can access Gemini's features through high-level APIs without needing to handle prompt-engineering, fine-tuning, capacity, or cost processes.
- DevTools: Gemini will understand errors and warnings on your sites, creating insights in the Chrome DevTools Console to help you understand and fix problems. Console insights are currently available as an experimental feature in the US and will roll out to more countries soon.
Speculation Rules API
- Speculation Rules will increase website speed by pre-fetching and pre-rendering pages in the background, allowing pages to load in milliseconds. You only need a few lines of code to start using this API.
View Transitions API
- With Chrome Canary 216, you can now use the View Transitions API for multi-page apps, previously limited to single-page apps.
Cross-Platform
Project IDX
- Project IDX is now open to the public as a beta. It’s an AI-assisted workspace for full-stack, multiplatform app development in the cloud.
- You have access to a dozen pre-loaded templates and also you can import your existing GitHub repository.
- You can debug issues with Chrome Web apps using DevTools and Lighthouse.
- You can enable multi-region deployment with Cloud Run.
- Google Maps Platform now makes it easier to add geo functionality to your apps.
Checks
- By using Google’s AI compliance platform Checks, you can simplify your app's privacy and compliance workflow.
- Checks Code Compliance monitors and detects compliance issues as you write code, helping ensure the safety of your applications.
- iOS and Android developers can now access Checks.
Flutter
- WASM support for Flutter Web apps is available today in the stable channel as part of Flutter 3.22 and Dart 3.4.
- Benchmarks show that compiling Flutter web apps to WASM offers up to a 2x improvement in frame time compared to Flutter web apps compiled to JavaScript.
Firebase
SQL Database
- You can now use SQL databases in Firebase through Firebase Data Connect and Cloud SQL, which are currently in preview.
- This offers a completely new approach to building secure and type-safe apps with Firebase.
- Your queries are automatically generated into type-safe client-side code, ensuring your app code stays in sync with your data structure.
- Data Connect is designed for AI development, featuring support for vector and function calling, allowing you to create AI agent-like flows and search for similarities using your stored embeddings.
- If you’re eager to try out Data Connect, you can sign up for the preview and explore one-click templates in IDX.
Hosting
- This next-generation serverless hosting product is currently in preview.
- App Hosting automatically pulls the source code from your GitHub repository, runs the framework build, and orchestrates the Google Cloud infrastructure you need.
Firebase GenKit
- Firebase GenKit is an open-source framework that helps you build, deploy, and monitor production-ready AI-powered apps, currently in beta. It provides APIs across various LLMs.
- Changing models is easy, with support for Gemini and Gemma.
- It includes local debug and trace support, allowing you to run GenKit locally on your RTX GPU with Ollama and Gemma.
- Use GenKit to create apps that generate custom content, use semantic search, handle unstructured inputs, answer questions with your business data, autonomously make decisions, orchestrate tool calls, and much more!
- GenKit currently supports server-side development in JavaScript/TypeScript (Node.js), with Go support in active development.
AI Models
- Created the Gemma family of open models for more flexibility and control, built on Gemini infrastructure, aiming for useful and responsible models.
- CodeGemma for coding tasks across languages.
- RecurrentGemma with recurrent neural networks and local attention for better memory efficiency and faster inference.
- PaliGemma for multimodal image-to-text tasks.
- Google AI Edge: On-device AI for mobile, web, and embedded applications.
Data Science Agent
- With Gemini 1.5Pro and its long context capability, you can upload a dataset and ask questions about it. The agent will complete the code, create a plan, execute it, and produce a fully functional Colab notebook based on your instructions. You can share this notebook with others for real-time collaboration on code, visualizations, and results. This feature is available now.
Google Developer Program
- Members of the program can now access new benefits at no cost:
- Access Gemini for learning, searching, and chatting with documentation.
- For IDX users, you can now go from two workstations up to five.
- If you’re in the Google Cloud Innovators community, you get credits for interactive labs on Google Cloud Skills Boost.
- More benefits coming soon.
Competition
- There is a competition where the grand prize is an electric classic DeLorean car.
