Why Is databricks-connect Not Recognised? One Error Message, Two Different Tools

An engineer installed the Databricks CLI on Windows with pip install databricks-cli. Pip reported Requirement already satisfied, so the install looked fine. Yet databricks-connect configure reported that the command was not recognised as a cmdlet, function, script file or operable program. Even databricks --version did nothing. The site-packages directory had already been added to the Windows PATH variable.

That is an awkward combination. Pip reports success and the shell reports that nothing is there. The obvious conclusion is that the installation is broken, and the obvious reflex is to reinstall. That does not help, because two unrelated problems are hiding behind the same message.

The accepted answer separated the two cleanly.

First, databricks-connect belongs to a different package. pip install databricks-cli provides the commands databricks and dbfs, and nothing else. Anyone who wants Databricks Connect installs it separately. The error message was simply correct: that command did not exist on the machine.

Second, the PATH entry was wrong. On Windows, pip places executables not in site-packages but in the Scripts directory of the Python installation, for example C:\Users\<username>\AppData\Roaming\Python\Python37\Scripts. As long as that directory is not in PATH, the shell will not find databricks, even though the package is genuinely there. Once the correct directory was added, databricks --version worked.

The follow-up made the source of the confusion clear. The original poster had seen databricks-connect in documentation while configuring the CLI, and assumed the two belonged together. The answer pointed out that the CLI is configured with databricks configure --token, not with databricks-connect configure.

Both pieces of advice were right for the tools of the time. Both have since been overtaken, and that is where the useful lesson sits.

The Databricks CLI is no longer a Python package. The documentation splits the versions into three groups: 1.0.0 and above is generally available, 0.205 through 0.299 is Public Preview, and 0.18 and below is the legacy CLI. That last group is exactly what pip install databricks-cli installs. Its documentation has been explicitly retired, it is no longer updated, it receives no new feature work, and it is not covered by Databricks Support.

The current CLI installs on Windows with WinGet (winget install Databricks.DatabricksCLI), with Chocolatey (experimental), through WSL with curl, or by extracting a zip file. The curl route places the file at C:\Windows\databricks.exe. None of those routes involves Python's Scripts directory. You verify it with databricks -v or databricks version; if that reports 0.205.0 or above, the installation is correct.

databricks-connect configure no longer exists either. That command belongs to Databricks Connect for Databricks Runtime 12.2 LTS and below, and that version is deprecated because all the corresponding LTS runtimes have reached end of support. For Databricks Runtime 13.3 LTS and above you install a package version that matches your cluster, for example pip3 install --upgrade "databricks-connect==17.3.*", and there is no configure command at all. There is also a step the thread misses: PySpark has to come out first, because databricks-connect conflicts with that package.

On one point the original poster was right without knowing it. The two tools do meet, at authentication rather than at commands. For OAuth user-to-machine, the documentation requires you to sign in with the Databricks CLI before you run your code. That happens through databricks auth login --host <workspace-url>, which writes a configuration profile that Databricks Connect then uses. From CLI 1.0.0 onwards those tokens land in the operating system's secure storage, which on Windows means Credential Manager, leaving .databrickscfg holding only non-secret configuration. Personal access tokens are now labelled legacy in the documentation, with OAuth as the recommendation.

Databricks Connect also imposes requirements that have nothing to do with PATH. Your account and workspace must have Unity Catalog enabled. The runtime version of your compute must be equal to or higher than the package version. A cluster must use an access mode of Assigned or Shared. Serverless compute is supported from Databricks Connect 15.1. And if you use user-defined functions, your local Python minor version has to match the one the runtime expects: Databricks Connect 17.3 requires Python 3.12, 15.4 requires 3.11 and 13.3 requires 3.10. The Python 3.7 from the original question meets none of those combinations.

The error message is about a missing executable. The question underneath it is which of two planes you actually needed.

The Databricks CLI sits on the control plane. It is a wrapper around the REST API and it manages objects: workspaces, jobs, clusters, DBFS and bundles. You use it to create, change and deploy things, and it does that just as well in a pipeline as on your laptop.

Databricks Connect sits on the data plane. It sends Spark code from your IDE to remote compute and brings the result back. You use it to execute work and step through it in a debugger, not to manage anything.

"The two tools share a brand name and almost nothing else. One manages the environment, the other uses it."

That distinction has a consequence that only becomes visible months later, which is why this is an architecture question rather than an installation question. The CLI is independent of your cluster versions. One binary serves every workspace, so IT can roll it out centrally and then forget about it. Databricks Connect is the exact opposite. Its package version is tied to the runtime version of your compute, and with user-defined functions your Python minor version is tied to it too. Upgrading a runtime therefore moves the development environment of every engineer working against that cluster.

That is also why they do not belong in the same installation. The CLI is a machine-wide tool. Databricks Connect belongs in a per-project virtual environment, and the PySpark conflict makes that more than a matter of taste. A shared Python installation where somebody needs PySpark for something else will break itself. Teams that treat both as "the Databricks toolchain" and put them in one environment are buying an upgrade problem.

The Unity Catalog requirement deserves separate attention, because it lifts the question out of the development toolchain. Databricks Connect for 13.3 LTS and above only works if the account and workspace have Unity Catalog enabled. On a workspace without Unity Catalog, "can I develop locally" is therefore not a tooling question but a governance question, and the answer sits with the platform team rather than with the engineer facing the PATH problem. The same applies to the cluster's access mode.

That makes the routing fairly simple. If you need to manage or deploy objects in the workspace, use the CLI, with Databricks Asset Bundles on top for deployments. If you need to run and debug Spark code from your IDE against remote compute, use Databricks Connect. If you just want to run a notebook, you need neither and you work in the workspace itself. The large majority of engineers genuinely need only the first or the third route, which is also the best antidote to this class of installation problem: install only what you use.

Two practical additions. If you work in Visual Studio Code, the Databricks extension can install Databricks Connect for you, which takes the version choice out of your hands. And one expectation is worth setting up front: Databricks Connect does not talk to an open source Apache Spark server, only to Databricks compute.