editors desk

A new version of the dynamically typed, high-performance Julia language for numerical computing has been released. Julia 1.11 features a new Memory type, a lower-level container that provides an alternative to Array. Downloadable from julialang.org, Julia 1.11 was released October 7 following two alphas, two betas, and four release candidates. Introduced with Julia 1.11, the Memory type has less overhead and a faster constructor than Array, making it a good choice for situations that do not need all the features of Array, according to release notes. Most of the Array type now is implemented in Julia on top of Memory as well, thus leading to significant speedups for functions such as push!, along with more maintainable code. Also in Julia 1.11, public is a new keyword. Symbols marked with public are considered public API, while symbols marked with export also are now treated as public API. The difference between export and public is that public names do not become available when using a package module. Additionally, tab completion has become more powerful and gains inline hinting when there is a singular completion available that can be completed with tab. Julia overall is billed as providing capabilities such as asynchronous I/O, metaprogramming, profiling, and a package manger. Other features in Julia 1.11: The entry point for Julia has been standardized to Main.main(args). The @time macro now will report any lock contention within the call being timed, as a number of lock conflicts. ScopedValue implements dynamic scope with inheritance across tasks. Manifest.toml files can now be renamed in the format Manifest-v{major}.{minor}.toml to be potentially picked up by the given Julia version. Code coverage and malloc tracking no longer are generated during the package pre-compilation stage. During these modes, pkgimage caches now are used for packages that are not being tracked. This means coverage testing will by default use pkimage caches for all other packages than the package being tested, likely meaning faster execution. At pre-compilation, atexit hooks now run before saving the output file, thus allowing users to safely tear down background state and clean up other resources when the program wants to start exiting. Specifying a path in JULIA_DEPOT_PATH now results in the expansion of empty strings to omit the default user depot. Pre-compilation cache files now are relocatable and their validity is verified through a content hash in their source files instead of their mtime. Unicode 15.1 is supported.

Every enterprise today is looking to get as much as possible out of its data and AI investments. But when building in-house apps, developers can struggle with infrastructure constraints and maintenance, security, data governance, compliance, and other issues. “Companies want to build custom experiences,” Shanku Niyogi, VP of product at Databricks, told InfoWorld. However, “as soon as you start to build a custom application, you start to fall off a cliff.” Databricks says its new platform, Databricks Apps, can help deal with these complexities. Today available in public preview on AWS and Azure, it says the platform allows users to create secure, tailored, enterprise-specific apps in just minutes. “Applications ultimately enable customers to really get value from their data and all their AI investments,” said Niyogi. Easy to build, deploy on an open, secure network Internal data applications have a variety of challenges, Hyoun Park, CEO and chief analyst at Amalgam Insights, told InfoWorld. Building data governance and controls is always a “major effort,” and apps need to be written in a language and with frameworks that can be supported on an ongoing basis. Companies need to worry about servers and cloud computing resources, and also must determine how to choose the right model for each use case while supporting customization, prompt engineering, and model augmentation. “Model flexibility has become an increasingly challenging aspect of data apps, especially for companies that have traditionally only built data apps to support traditional analytics and reporting use cases,” Park explained. In contrast, Databricks Apps is easy to build and deploy, and has an “open approach,” with Python as its primary language, Niyogi explained. If users know Python, they can build an app in as few as 5 minutes, he said. The platform provides automated serverless compute, meaning users don’t need IT teams to set up infrastructure. It supports Dash, Shiny, Grado, Streamlit and Flask frameworks, and apps are automatically deployed and managed in Databricks or in a user’s preferred integrated development environment (IDE). “You can build, fine tune, train, and serve ML (machine learning) models on top of your data directly inside Databricks,” said Niyogi. To support security, data never leaves Databricks, he explained, and the app is managed by the company’s data governance tool, Unity Catalog. All users are authenticated through OIDC/ OAuth 2.0 and single sign on (SSO). “Securing applications can become very difficult,” said Niyogi, as users have to manage controls and add credentials. “It’s often pretty brittle, difficult to manage.” With Databricks Apps, multiple layers of security, including for physical infrastructure such as VPNs, help to ensure that data doesn’t leave the compliance and regulatory boundary. “You share data when you need to,” said Niyogi. Further, lineage tracking in Unity Catalog allows visibility into what apps and users are accessing what data, and who is making changes. With this integrated security, some customers have been able to get their first apps into production in days, instead of waiting for weeks for security teams to perform reviews, Niyogi reported. “Databricks Apps takes advantage of Databricks’ native capabilities to support enterprise-grade data governance and to trace data back to its original source,” said Park. By choosing a serverless deployment method, Databricks doesn’t constrain app storage or compute. The user experience is also “fairly straightforward,” with standard Python frameworks and templates, he said. “This experience is not unique, but provides parity with other development environments,” said Park. He pointed out that Databricks Apps has a lot of competition from business intelligence vendors supporting data apps such as Tableau, Qlik, Sisense and Qrvey. It also vies for market share with “mega vendors” including Microsoft, Oracle, SAP, Salesforce, ServiceNow, and Zoho. Then there are low-code and no-code apps such as Mendix, Appian, and Quickbase at the fringes of the market. The most important “tactical capabilities” Databricks brings to the table with the new platform, Park noted, is the ability to reuse existing governance, launch from an open-ended serverless environment, and provide a single tool to manage data, infrastructure, and code applications all at once. “This announcement is consistent with the current brand promise of Databricks as a ‘data intelligence platform’ rather than just being a data insight or data discovery platform,” said Park. Interfaces that map to data models Ideal use cases for Databricks Apps include AI applications, analytics, data visualization, and data quality monitoring, said Niyogi. For instance, a marketing team may create customized dashboards to visualize campaign performance metrics. They could also incorporate AI to perform sentiment analysis on customer feedback, or predictive modeling for forecasts, customer segmentation, or fraud detection. Databricks customer SAE International, for example, used the platform to turn a retrieval augmented generation (RAG) proof of concept into a branded application that answers questions based on the aerospace company’s knowledge base. And IT services and consulting company E.ON Digital Technology incorporated the platform into its DevSecOps processes to test new features. Other early users include open-source data science company Posit and data apps platform Plotly. “You can build an interface that really maps to your data model,” said Niyogi. As Databricks has adopted the platform internally, its own employees have built tools to help improve personal self-management, he said. “Application building is fun,” said Niyogi. “AI development tools unlock people’s creativity. We’re excited to see what our customers build.”

In a significant development in the cloud computing industry, AWS, a leading hyperscaler, has recently acknowledged considerable competition from on-premises IT solutions. This admission counters the longstanding belief that enterprises seldom revert to on-premises systems once they transition to the cloud. AWS’s statement emerged during a Competition and Markets Authority hearing in the United Kingdom, which scrutinized the current state of competition within the cloud market. At the hearing, AWS argued that the prevailing perception must accurately reflect the dynamics of the IT services market. This was obvious to me, and I’m unsure why it had to be said. Keep in mind that there could be other motivations here. They may be spinning the market so they won’t be concerned with monopolistic players. Europe seems to be looking at the role of public cloud providers with a concern that they may have too much control. If AWS is pointing to traditional solutions as a core alternative solution that companies are using, they are not a monopoly. No public cloud providers are at this point in the market. A complex cloud market This is not an AWS issue but a broader cloud market issue that should be analyzed and understood. AWS behaves pretty much like any public cloud provider, examining the market dynamics of 2025 and strategizing how to grow the cloud business best. The trends are away from a “cloud-or-die” approach to IT. All platforms are sound architectural options, including on-premises and public cloud. It took many of us 10 years to understand this. To me, it’s just common sense. The cloud market is inherently complex, and cloud providers often emphasize their own solutions’ cost efficiency and innovation capabilities. Nonetheless, cloud providers have denied any allegations suggesting their practices create technical barriers or impose unfair financial burdens on customers, such as egress fees or committed spend discounts. They insist that the cloud services market operates well and meets the needs of U.K. and global customers concerning pricing, innovation, and quality of service. Despite AWS’s confidence in the competitive fairness of the cloud market, criticism has been directed toward its claims of significant competition from on-premises migrations. However, for many organizations, on-premises and private cloud solutions provide a more reliable guarantee of data sovereignty, an increasingly crucial consideration in today’s data-driven landscape. The issues to consider The resurgence of on-premises solutions as a competitive option arises from several factors influencing enterprise IT strategies. Cost management. Although cloud solutions have been marketed as cost-efficient, particularly with their pay-as-you-go models, the reality can differ. Over time, the cumulative costs of cloud services are often higher than on-premises systems. This is undoubtedly the case for stable workloads that don’t benefit from the elasticity of the cloud. Companies seeking predictable expenses and long-term savings may find value in reverting to or maintaining on-premises infrastructure. Cheap hardware. Real cheap. I pointed this out about 100 times in my last book. This means the compelling reason to leverage public clouds (cost) is tossed out the window. Storage costs have dropped at a 45-degree angle in the past 10 years, to the point that many companies are planning repatriation projects to take advantage. I’ve done a few dozen myself. Data security and sovereignty. Regulations, such as General Data Protection Regulation (GDPR) in Europe, are causing concern for many enterprises. On-premises systems give companies more control over their data, reducing the risk of breaches and ensuring compliance with local data protection regulations. Performance and control. Some applications require high performance and low latency, especially in industries such as finance or gaming. These systems may benefit from the proximity and control of on-premises infrastructure. This setup allows for more fine-tuning and optimization, which public clouds might not offer. Customizability and flexibility. On-premises systems often provide more significant opportunities for customization. Enterprises with specific infrastructure needs can tailor their on-premises systems more precisely than they might be able to with generic cloud services. Technological advancements. Recent innovations in on-premises technologies, such as hyperconverged infrastructure and enhanced virtualization capabilities, have made setting up and managing an on-premises data center more efficient. Hybrid and multicloud strategies. Many enterprises are blending both on-premises and cloud resources to optimize their IT environments. This approach offers the benefits of both worlds: on-premises for stable, predictable workloads, and cloud for scalable, dynamic needs. Vendor dissatisfaction. Enterprises are not happy with major cloud providers due to service outages, egress fees, or lack of transparency around pricing and service-level commitments. Businesses are rethinking their cloud reliance and exploring on-premises alternatives. A new way of thinking I’ve often pointed out that the IT world is moving to heterogeneity and ubiquity, meaning that no approach, cloud or on-premises, will rise to the top and become “the standard.” I view this as a good thing, as long as we’re prepared to deal with the complexity it will bring. Thus far, many enterprises have stubbed their toe on that issue. I’m now having these conversations often, whereas in the past, it was not spoken about in polite company–and certainly not at cloud conferences. Indeed, the concepts of multicloud and optimization are seeping back into the talks at significant events, when just a few years ago, presenters were pulling those slides out of their decks. These new conversations reveal a nuanced picture of enterprise IT strategies, where flexibility and adaptability are paramount. Organizations are increasingly discerning, weighing the prowess of cloud computing against the tangible benefits of maintaining or reverting to on-premises solutions. This dynamic landscape signifies a transformative period for the IT industry, compelling cloud providers to continuously innovate and align with customer expectations to retain their competitive edge. We’ll see what 2025 brings.

Artificial intelligence has the power to revolutionize industries, drive economic growth, and improve our quality of life. But like any powerful, widely available technology, AI also poses significant risks. California’s now vetoed legislation, SB 1047 — the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act — sought to combat “catastrophic” risks from AI by regulating developers of AI models. While lawmakers should be commended for trying to get ahead of the potential dangers posed by AI, SB 1047 fundamentally missed the mark. It tackled hypothetical AI risks of the distant future instead of the actual AI risk of today, and focused on organizations that are easy to regulate instead of the malicious actors that actually inflict harm. The result was a law that did little to improve actual safety and risks stifling AI innovation, investment, and diminishing the United States’ leadership in AI. However, there can be no doubt that AI regulation is coming. Beyond the EU AI Act and Chinese laws on AI, 45 US states introduced AI bills in 2024. All enterprises looking to leverage AI and machine learning must prepare for additional regulation by boosting their AI governance capabilities as soon as possible.  Addressing unlikely risks at the cost of ignoring present dangers There are many real ways in which AI can be used to inflict harm today. Examples of deepfakes for fraud, misinformation, and non-consensual pornography are already becoming common. However, SB 1047 seemed more concerned with hypothetical catastrophic risks from AI than with the very real and present threats that AI poses today. Most of the catastrophic risks envisioned by the law are science fiction, such as the ability of AI models to develop new nuclear or biological weapons. It is unclear how today’s AI models would cause these catastrophic events, and it is unlikely that these models will have any such capabilities for the foreseeable future, if ever.  SB 1047 was also focused on commercial developers of AI models rather than those who actively cause harm using AI. While there are basic ways in which AI developers can ensure that their models are safe — e.g. guardrails on generating harmful speech or images or divulging sensitive data — they have little control over how downstream users apply their AI models. Developers of the giant, generic AI models targeted by the law will always be limited in the steps they can take to de-risk their models for the potentially infinite number of use cases to which their models can be applied. Making AI developers responsible for downstream risks is akin to making steel manufacturers responsible for the safety of the guns or cars that are manufactured with it. In both cases you can only effectively ensure safety and mitigate risk by regulating the downstream use cases, which this law did not do.     Further, the reality is that today’s AI risks, and those of the foreseeable future, stem from those who intentionally exploit AI for illegal activities. These actors operate outside the law and are unlikely to comply with any regulatory framework, but they are also unlikely to use the commercial AI models created by the developers that SB 1047 intended to regulate. Why use a commercial AI model — where you and your activities are tracked — when you can use widely available open source AI models instead?   A fragmented patchwork of ineffective AI regulation Proposed laws such as SB 1047 also contribute to a growing problem: the patchwork of inconsistent AI regulations across states and municipalities. Forty-five states introduced, and 31 enacted, some form of AI regulation in 2024 (source). This fractured regulatory landscape creates an environment where navigating compliance becomes a costly challenge, particularly for AI startups who lack the resources to meet a myriad of conflicting state requirements.  More dangerous still, the evolving patchwork of regulations threatens to undermine the safety it seeks to promote. Malicious actors will exploit the uncertainty and differences in regulations across states, and will evade the jurisdiction of state and municipal regulators. Generally, the fragmented regulatory environment will make companies more hesitant to deploy AI technologies as they worry about the uncertainty of compliance with a widening array of regulations. It delays the adoption of AI by organizations leading to a spiral of lower impact, and less innovation, and potentially driving AI development and investment elsewhere. Poorly crafted AI regulation can squander the US leadership in AI and curtail a technology that is currently our best shot at improving growth and our quality of life. A better approach: Unified, adaptive federal regulation A far better solution to managing AI risks would be a unified federal regulatory approach that is adaptable, practical, and focused on real-world threats. Such a framework would provide consistency, reduce compliance costs, and establish safeguards that evolve alongside AI technologies. The federal government is uniquely positioned to create a comprehensive regulatory environment that supports innovation while protecting society from the genuine risks posed by AI. A federal approach would ensure consistent standards across the country, reducing compliance burdens and allowing AI developers to focus on real safety measures rather than navigating a patchwork of conflicting state regulations. Crucially, this approach must be dynamic, evolving alongside AI technologies and informed by the real-world risks that emerge. Federal agencies are the best mechanism available today to ensure that regulation adapts as the technology, and its risks, evolve. Building resilience: What organizations can do now Regardless of how AI regulation evolves, there is much that organizations can do now to reduce the risk of misuse and prepare for future compliance. Advanced data science teams in heavily regulated industries — such as finance, insurance, and healthcare — offer a template for how to govern AI effectively. These teams have developed robust processes for managing risk, ensuring compliance, and maximizing the impact of AI technologies. Key practices include controlling access to data, infrastructure, code, and models, testing and validating AI models throughout their life cycle, and ensuring auditability and reproducibility of AI outcomes. These measures provide transparency and accountability, making it easier for companies to demonstrate compliance with any future regulations. Moreover, organizations that invest in these capabilities are not just protecting themselves from regulatory risk; they are positioning themselves as leaders in AI adoption and impact. The danger of good intentions While the intention behind SB 1047 was laudable, its approach was flawed. It focused on organizations that are easy to regulate versus where the actual risk lies. By focusing on unlikely future threats rather than today’s real risks, placing undue burdens on developers, and contributing to a fragmented regulatory landscape, SB 1047 threatened to undermine the very goals it sought to achieve. Effective AI regulation must be targeted, adaptable, and consistent, addressing actual risks without stifling innovation. There is a lot that organizations can do to reduce their risks and comply with future regulation, but inconsistent, poorly crafted regulation will hinder innovation and will even increase risk. The EU AI Act serves as a stark cautionary tale. Its sweeping scope, astronomical fines, and vague definitions create far more risks to the future prosperity of EU citizens than it realistically limits actors intent on causing harm with AI. The scariest thing in AI is, increasingly, AI regulation itself. Kjell Carlsson is the head of AI strategy at Domino Data Lab, where he advises organizations on scaling impact with AI. Previously, he covered AI as a principal analyst at Forrester Research, where he advised leaders on topics ranging from computer vision, MLOps, AutoML, and conversation intelligence to next-generation AI technologies. Carlsson is also the host of the Data Science Leaders podcast. He received his Ph.D. from Harvard University. — Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact doug_dineley@foundryco.com.

SAP wants developers to view its Build platform as the one extension solution for all of SAP’s applications, according to Michael Aneling, chief product officer for SAP Business Technology Platform (BTP). At its TechEd event this week it is showing a new extensibility wizard, now available, that gives developers access to SAP Build directly from S/4HANA Cloud Public Edition, enabling them to extend custom fields, business logic, and processes either within the S/4HANA Cloud ABAP environment or in SAP BTP. “A critical feature of this integration is the knowledge of business context — such as events and objects — when creating business processes and while transitioning between SAP S/4HANA and SAP Build,” SAP noted. “This preserves important business information, letting developers seamlessly switch between environments without losing crucial context.” By year-end, the extensibility wizard will enable developers to create custom SAP Fiori and SAPUI5 applications as well. Around the same time, SAP will release Joule studio in SAP Build, which will provide a dedicated environment for businesses to create, deploy, monitor and manage custom skills for its AI copilot, Joule. Custom skills, which complement Joule’s out-of-the-box capabilities, will extend conversational AI to organization-specific workflows. Third-party AI integrations Additionally, customers will be able to integrate any third-party system into Joule to create what SAP described as “an entirely integrated conversational user experience.” Joule will also be integrated into SAP Build Work Zone, a low-code tool for creating web sites. Joule’s generative AI capabilities will provide support while navigating data from connected business systems. All this will be available in SAP Build Work Zone standard edition, the SAP Start site, and the SAP Mobile Start app. New capabilities such as code explanation and documentation search in SAP Build Code will assist Java and JavaScript developers, who will also be able to automate workflows in SAP Build Process Automation, with assistance from generative AI. Early next year, SAP plans to extend Joule to help developers using ABAP (Advanced Business Application Programming), SAP’s high-level programming language, to generate high-quality code and unit tests that comply with SAP’s ABAP Cloud development model. Joule will also be able to generate explanations for legacy code, to ease the path to modernizing legacy codebases and the migration to a “clean core”, a modern ERP system without hard-coded customizations. Generative AI hub updates By the end of the year, 2024, developers will be able to customize pre-trained AI models using a guided process, to create their own AI-driven applications, SAP said. They will also have the ability to integrate advanced AI capabilities into web applications, with new software development kit (SDK) support for ABAP, Java, and JavaScript. This toolkit will help them embed intelligent features like chatbots and content generators into sites. Another new SDK, the ABAP AI SDK, will let developers access generative AI hub capabilities from within custom ABAP applications to allow them to add AI functions to those applications and extensions. New LLMs Finally, SAP is adding several new large language models (LLMs) to the generative AI hub by the end of this year, raising the number to more than 30. The additions include: Aleph Alpha Pharia-1 Amazon Titan Image Generator IBM Granite Mistral Large 2 OpenAI Dall-E 3

Oracle is touting a “tip and tail” model for Java library development that the company says would give application developers a better experience and at the same time help library developers innovate faster. The JEP (JDK Enhancement Proposal) created September 30 and updated October 7 describes a release model for software libraries. The “tip” release of a library contains new features and bug fixes, the proposal states, while “tail” releases contain only critical bug fixes. As little as possible is backported from the tip to the tails. The JDK itself has used tip and tail since 2018 to deliver new features at a faster pace and provide reliable, predictable updates focused on stability. Goals of the plan include: Helping the Java ecosystem maintain the balance between innovating rapidly for new development and ensuring stability for long-term developments. Recognizing that application developers have diverse views about changes to make it necessary to update libraries and the JDK. Ensuring library developers do not have to choose between supporting users of older JDKs and embracing new features, such as virtual threads and patterns, that excite users of newer JDKs. Not constraining library release cycles, version schemes, or bad choices. In explaining the motivation behind the proposal, the proposal states that the tip and tail model is a streamlined form of the multi-train model, which lets libraries serve a diverse user base while embracing new Java features. Tip and tail gives users focused on stability what they need, namely fixes and patches, while giving users building new systems what they want, namely features and enhancements, at a faster pace. The tip and tail model will keep the Java platform an attractive choice for new applications while safeguarding the future of existing applications, the proposal stresses.

Rust, the fast and memory-safe programming language born out of Mozilla, has resumed its rise up the monthly Tiobe index of programming language popularity, although it still has not cracked the index’s top 10. Rust returned to 13th place in the October edition of the Tiobe index, the highest spot the language has reached. Rust had ranked 13th in July before slipping to 14th place in August. An emphasis on security and speed is helping boost Rust’s place, even though the language is not easy to learn, said Paul Jansen, CEO of software quality services company Tiobe. Rust contrasts with index leader Python, which is easy to learn and secure but not fast, Jansen said. Jansen believes Rust is making its way to Tiobe’s top 10. The Tiobe index rates programming languages based on the number of skilled engineers worldwide, courses offered, and third-party vendors pertinent to each language. The ratings are calculated by examining popular websites such as Google, Amazon, Wikipedia, and Bing. The rival Pypl Popularity of Programming Language index already places Rust in the top 10. Once again finishing 10th in Pypl’s index for October, Rust has ranked 10th consistently since April 2024. The Pypl index assesses language popularity based on how often language tutorials are searched on Google. Elsewhere in the Tiobe index for October, the Mojo language, described by Jansen as a mix of Python and Swift but much faster, entered the top 50 for the first time. It ranks 49th. “The fact that this language is only one year old and already showing up, makes it a very promising language,” Jansen said. The Tiobe index top 10 for October 2024: Python, 21.9% C++, 11.6% Java, 10.51% C, 8.38% C#, 5.62% JavaScript, 3.54% Visual Basic, 2.35% Go, 2.02% Fortran, 1.8% Delphi/Object Pascal, 1.68% The Pypl index top 10 for October 2024: Python, 29.56% Java, 15.66% JavaScript, 8.16% C/C++, 6.76% C#, 6.58% R, 4.64% PHP, 4.2% TypeScript, 2.95% Swift, 2.64% Rust, 2.55%

Python 3.13 has just been released. This article presents a rundown of the most significant new features in Python 3.13 and what they mean for Python developers. Major new features in Python 3.13 Here’s a rundown of the biggest new features in Python 3.13: The experimental JIT The no-GIL build of Python A new REPL Improved error messages Enhancements to Python types No more “dead batteries” The experimental JIT Python 3.11 introduced the Specializing Adaptive Interpreter. When the interpreter detects that some operations predictably involve the same types, those operations are “specialized.” The generic bytecode used for that code is swapped with bytecode specific to working with those types, which delivers speed boosts of anywhere from 10% to 25% for those regions of the code. Python 3.12 brought more specializations and other refinements to the interpreter. Now, Python 3.13 adds new elements to the JIT that generate actual machine code at runtime, instead of just specialized bytecode. The resulting speedup isn’t much just yet—maybe 5%—but it paves the way for future optimizations that weren’t previously possible. Right now, the JIT is considered experimental—it’s not enabled by default, and can only be enabled by compiling CPython from source with certain flags. If in time it yields a significant performance boost (5% or more), and doesn’t impose a large management burden on the CPython team or Python’s users as a whole, it’ll become a fully supported build option. Whether or not it will be enabled for official releases will still be up to the managers for a given platform’s CPython builds. Python’s release cycle The Python programming language releases new versions yearly, with a feature-locked beta release in the first half of the year and the final release toward the end of the year. Developers are encouraged to try out this latest version on non-production code, both to verify that it works with your programs and to get an idea of whether your code will benefit from the new feature sets and performance enhancements in this latest version. The no-GIL ‘free-threaded’ build of Python The official term for possible future versions of CPython with no Global Interpreter Lock (or GIL) is “free-threaded CPython.” This CPython build allows threads to run fully in parallel, without mediation from the GIL. To that end, CPU-bound work that once only benefited from being run in multiple processes can run in multiple threads. Free-threaded CPython is also experimental. It’s not enabled by default in the shipped builds, so it needs to be enabled at compile time. If future work with the free-threaded builds shows it can improve multithreaded performance without impacting single-threaded performance, it’ll be promoted to a fully supported option. In time, the free-threaded build of CPython may become the default. A new REPL The REPL, or interactive interpreter, launches when you run Python from the command line without executing a program. Python 3.13’s REPL has enhancements to make it less stodgy and more like an actual editor: Output to the console now has color enabled by default. This enhancement provides richer error messages, for instance. You can open the interactive pydoc help browser by pressing F1. You can browse the command-line history with F2. You can paste large blocks of code more easily by pressing F3 to enable a special block-paste mode. You can just type exit or quit, instead of exit() or quit(), to leave the REPL. Note that these improvements currently are only available on Linux and macOS. They are not available on Microsoft Windows, not even when using the new Windows Terminal console host. Improved error messages Error traces in Python have become more precise and detailed over the last two releases. Python 3.13 continues on that trajectory. If you attempt to import something that has the same name as the module currently in context, Python will provide a detailed error to that effect, and encourage you to rename the current module. This is a very frequent source of bugs—and not only for beginners. It’s a common mistake to name a module after something in the standard library. If you pass a function an incorrect keyword argument, the error will suggest some possible correct arguments, based on what’s available in the function being called. Where supported, error messages now use color in tracebacks to make them easier to read. Enhancements to Python types Python’s type hinting system has expanded in functionality and utility with each new version. Version 3.13 adds three big new changes. Type parameters support defaults typing.TypeVar, typing.ParamSpec, and typing.TypeVarTuple all let you define defaults to be used if no type is explicitly specified. For instance: T = TypeVar("T", default=str) In cases where T is not explicitly defined when used, str is assumed to be the default. typing.TypeIs for type narrowing In Python generally, we can use isinstance() to make decisions based on whether or not something is a given type. typing.TypeIs lets us do the same thing in Python’s type hinting mechanisms. This way, functions used to validate whether or not something is a given type can be annotated to show they perform that narrowing behavior, rather than just a return type. This is useful as a way to add precise type checker coverage to those functions. typing.ReadOnly for read-only annotation The typing.TypedDict type was created to annotate dictionaries with fixed types for the values associated with certain keys. typing.Readonly lets you annotate specific values in a TypedDict as read-only. An example is a list that you can only append to or pop from, not replace with a string or other type. No more ‘dead batteries’ Python 3.11 identified a slew of Python standard library modules that were obsolete and no longer being maintained. The plan was to mark them as deprecated for 3.11 and 3.12, and then remove them entirely in Python 3.13. As of now, those “dead batteries” (as they’ve been called) are now permanently removed. Many of the removed modules can be replaced with third-party modules, or their functionality can be emulated using other standard library components. Users can expect more deprecations to come over the next three versions of Python, as well. Most are methods for various standard library components that are rarely used or undocumented.

“You don’t want AI only in the hands of a few companies.” Thus spake Hugging Face CEO Clem Delangue, suggesting that open source will come to the rescue of AI. It’s a nice thought, but one that has exactly zero historical precedent. Yes, open source has become integral to building software, but name a market where open source has prevented that market from settling into “the hands of a few companies.” Go ahead. I’ll wait. Cloud? Years ago I wrote that cloud is impossible without open source, and I still believe that. But there are only a few big winners in cloud infrastructure. Likewise there are only a few big winners in any particular category of SaaS, only a few big winners in…you get the point. Open source may enable big markets, but it doesn’t enable widespread spoils from those markets, because ultimately people—and enterprises—pay for someone to remove the complexity of choice. So, by definition, there can only be a few “someones” in any given market. Open source enablement Back to Delangue, who says, “I think open source comes in as a way to create more competition, to give more organizations and more companies the power to also build AI, build their own system that they control, to make sure that they don’t only rely on big technology companies.” He may be right that open source creates more opportunity to build for more companies, but he’s completely wrong that people won’t end up depending on big technology companies. I don’t say that out of some desire that this is how it should be—it’s just how things actually are. Again, look at cloud. Lots of open source hasn’t diffused control of the cloud market. If anything, it has concentrated it. With so much open source available, enterprises have needed cloud companies to help them make sense of it all. Enterprises haven’t really cared about the provenance of that open source code, either. After all, the biggest winner in cloud (Amazon Web Services) has, to date, been the smallest contributor to open source, relatively speaking. That has changed in the past few years, with AWS contributing across a swath of projects, from Postgres to OpenTelemetry to Linux. My point isn’t to criticize AWS. Not at all. After all, AWS has done what customers want: made all that open source easily consumable by enterprises, whatever its source. We can wish that AI will be different, but it’s hard to see how. The winners in AI As Richard Waters notes in the Financial Times, “OpenAI’s biggest challenge [is] the lack of deep moats around its business and the intense competition it faces.” That competition isn’t coming from open source. It’s coming from other well-capitalized businesses—from Microsoft, Meta, and Google. One of the biggest issues in AI right now is how much heavy lifting is imposed on the user. Users don’t want or need a bunch of new, open source–enabled options. Rather, they need someone to make AI simpler. Who will deliver that simplicity is still up for discussion, but the answer isn’t going to be “lots of open source vendors,” because, by definition, that would simply exacerbate the complexity that customers want removed. Yes, we should be grateful for open source and its impact on AI, just as we should for its impact on cloud and other technology advances. But open source isn’t going to democratize AI any more than it has any other market. The big thing that customers ultimately care about, and are willing to pay for, is convenience and simplicity. I still believe what I wrote back in 2009: “No one cares about Google because it’s running PHP or Java or whatever. No one cares about the underlying software at all; at least, its users don’t.”

Until recently, data scientists and analysts’ primary deliverables were data visualizations, machine learning models, dashboards, reports, and analytical insights used for storytelling. Now, with genAI capabilities, data scientists are called to expand their analytics to include unstructured data sources, help business teams pivot to data-driven decision-making, consult on AI ethics and governance, and help establish guardrails for the growing ranks of citizen data scientists. “GenAI accelerates time-to-insight, lowers technical skills barriers, and empowers teams to scale bandwidth for data-driven decision making,” says Anant Adya, EVP at Infosys Cobalt. “While human expertise remains crucial, genAI acts as a potent force multiplier, augmenting human capabilities and unlocking new data innovation opportunities.” In a recent OpenText survey on AI and analytics conducted by Foundry, 75% of respondents said leveraging genAI for data visualization and reporting was important. However, only 27% of respondents in data architecture and analytics roles ranked it as critically important. AI is driving significant business expectations, and leaders expect data scientists and analysts to gain the knowledge and skills needed to deliver competitive advantages. Data science teams should review their goals and discuss their strategies for leveraging generative AI. “Analytics, data visualization, and machine learning are rapidly advancing with generative AI capabilities, enabling more intuitive data interactions, automated insights, and sophisticated predictive models,” says Sreekanth Menon, global head of AI/ML at Genpact. “As these technologies evolve, generative AI enhances these fields by creating more accurate visualizations, simplifying complex data interpretation through natural language processing, and automated generation of analytic reports.” I’ve recently covered how data governance, software development, low-code development, and devsecops are evolving in response to AI breakthroughs and new business drivers. This article looks at the evolution of data scientists’ and analysts’ roles and responsibilities and the tools and processes they use. Target revenue and growth Data scientists have always sought a portfolio of use cases to apply their skills to, including lead generating in marketing, pipeline optimization for sales, profitability analysis for finance, and skills development for human resources. Finding productivity improvements is important, but with genAI, data scientists should expect greater demand for their services, especially in revenue growth areas, as businesses seek new digital transformation opportunities leveraging AI. “To go beyond mere productivity gains, it’s important to focus on accelerating long-tail revenue, which has already benefited from digital transformation but still relies on human analysis. AI can now enhance this area for greater topline growth,” says Sreedhar Kajeepeta, CTO at Innova Solutions. “Key areas include analyzing demand from long-tail customers to adjust products and services, optimizing pricing and promotions, creating targeted marketing content for niche segments, and identifying new customer segments beyond traditional sales strategies.” Paul Boynton, co-founder and COO at Company Search Incorporated (CSI), adds these strategic analytics use cases. “Generative AI significantly enhances the user interface for analyzing market trends, predicting product demand, optimizing supply chain efficiency, and identifying compatible partnerships to drive sales and growth,” he says. To meet these increased business needs, data scientists will need to increase their business acumen and find ways to discover and analyze new data sets targeting revenue growth. Integrate with AI-generated dashboards Data scientists have traditionally developed dashboards as quick and easy ways to learn about new data sets or to help business users answer questions about their data. While data visualization and analytics platforms have added natural language querying and machine learning algorithms over the last several years, data scientists should anticipate a new wave of genAI-driven innovations. “Over the next two years, we expect a transition from static business intelligence dashboards to more dynamic, personalized analytics experiences,” says Alvin Francis, VP of product management for business analytics at IBM. “With generative AI, the reliance on traditional dashboards diminishes as users can remove the noise of the analytics and get to actionable insights conversationally. Freed from ad-hoc dashboard-generation, data analysts and data scientists will concentrate on documenting organizational knowledge into semantic layers and conducting strategic analytics, creating a virtuous cycle.” Another prediction comes from Jerod Johnson, senior technology evangelist at CData, saying, “As genAI platforms become integrated into visualization tools, they enable more dynamic and interactive representations of data, allowing for real-time synthesis and scenario analysis. Over the next few years, data scientists can expect these tools to evolve to make visualizations more intuitive and insightful, even answering unasked questions for innovative discoveries.” Data scientists should use this period to learn how to use genAI capabilities in their data visualization platforms. As visualization becomes easier, data scientists will need to be prepared to use the advanced analysis capabilities to deliver new types of insights. Empower citizen data scientists Many leaders expect an uptick in features targeting citizen data scientists and an increase in business people learning self-service business intelligence tools with genAI capabilities. “GenAI is unlocking data’s full potential, enabling IT professionals to optimize planning and analytics capabilities through extended functionalities and automated workflows,” says Jared Coyle, head of AI at SAP North America. “This evolution streamlines complex tasks and makes advanced tools more accessible to non-technical users. In the coming years, increased automation of routine tasks will empower teams to focus on more strategic work, driving more efficient data-driven decisions across organizations.” The growth is likely to come as data visualization tools enhance natural language capabilities and automate the application of machine learning models. These capabilities will simplify the work for citizen data scientists, who can query data, find outliers, identify trends, and create and maintain dashboards with less expertise and fewer clicks. “GenAI-powered applications and platforms can generate dynamic visualizations, data storytelling narratives, and clear explanations for complex data insights,” says Sharmodeep Sarkar, enterprise AI architect at RR Donnelley. “This makes them easier to understand for non-technical audiences, helping to breed empowered ‘citizen data analysts’ across large organizations.” The transition of data, analytics, visualization, and modeling skills from data science to business teams has been happening for over a decade, but genAI is likely to be an accelerant. What does this mean for data scientists and their work? “As GenAI becomes more integrated in analytics, routine tasks like data prep and basic analysis will become more automated, freeing up time to dive deeper into insights,” says Jozef de Vries, chief product engineering officer at EDB. “Advanced AI tools will make data visualization and storytelling more intuitive, making it easier for data scientists to communicate complex findings to non-technical colleagues while also empowering these colleagues to use natural language to explore data. This will help bridge the gap between data teams and other departments, fostering a more collaborative environment.” Julian LaNeve, CTO at Astronomer, says data science teams should expect increased stakeholder interest and participation because of generative AI capabilities. “The barrier to entry to interact with and extract insights from data will be significantly lower, so establishing a strong data culture and practices is extremely important,” he says. LaNeve recommends developing a proper data platform based on data engineering best practices and well-cataloged data dictionaries for non-technical colleagues. Another role is consulting on proper governance and guardrails for end users. Harness unstructured data sets As analyzing rows and columns of data becomes easier for business users, data scientists should expand their skills and analytics efforts to investigate unstructured data sources. Many marketing, sales, and customer service data sets are unstructured, so analyzing them helps align with businesses that seek growth and competitive advantage. “Generative AI is revolutionizing how customer-centric organizations synthesize and analyze large volumes of free-text conversations,” says Saeed Aminzadeh, CPO at mPulse. “By accurately categorizing consumer intent and needs at scale, these advanced tools provide richer, more actionable insights.” One technology data scientists should learn is graph databases. Another, knowledge graphs, can be useful for developing RAGs that augment LLM models with domain intelligence. “Organizing data as knowledge graphs instead of flat SQL tables gives a tremendous advantage in doing advanced analytics, but also running machine learning models,” says Nikolaos Vasiloglou, VP of research ML at RelationalAI. “The most frequent task is feature engineering, and as LLMs get embedded in knowledge graphs, data scientists should expect to get more meaningful generated features.” Hema Raghavan, head of engineering and co-founder at Kumo AI, says data scientists should be familiar with graph neural networks (GNNs). “GNNs have the capability to look across tables and find the signal needed for predictive AI tasks, thus eliminating the need for a large number of feature engineering workflows. Data scientists can then focus on impact and identifying opportunities in their business where predictions can be plugged in.” Leverage AI agents and models Two emerging AI capabilities that should interest data scientists are industry-specific AI models and AI agents. For example, Salesforce recently announced Industries AI, a set of pre-built customizable AI capabilities that address industry-specific challenges across 15 industries, including automotive, financial services, healthcare, manufacturing, and retail. One healthcare model provides benefits verification, and an automotive model provides vehicle telemetry summaries. Regarding AI agents, Abhi Maheshwari, CEO of Aisera, says, “AI agents elevate LLMs by engaging in reasoning, planning, decision-making, and tool usage, handling tasks like CRM and ERP transactions autonomously. These agents simplify data tasks usually done by data analysts, including cleaning, exploratory data analysis, feature engineering, and forecasting.” These two trends illustrate a secondary shift in the data science role—from wrangling data and developing machine learning models to focusing on leveraging AI agents, investigating third-party models, and collaborating with citizen data scientists on applying AI, machine learning, and other data science capabilities. Another critical area for data scientists to become versed in is AI ethics and how that contributes to their organization’s AI governance. “As genAI embeds further in analytics, data science teams must adapt by acquiring new skills, focusing on strategic collaboration, and prioritizing AI ethics,” says Bogdan Raduta, head of AI at FlowX.AI. Menon of Genpact says, “The use of genAI in data storytelling will need to address ongoing challenges such as mitigating biases and ensuring the accuracy of generated content through responsible AI to ensure ethical use, transparency, and fairness, enhancing trust and accuracy in data-driven decision-making.” There is no doubt that AI is transforming how data scientists do their work and what tasks they focus on. The real opportunities lie in guiding the organization forward and delivering analytics-driven impacts in ethical ways.