Skip to main content

Welcome to Meroxa!

We are excited share the Meroxa platform and the Turbine data application framework. Just a few steps to get started and you're ready to build your first Turbine data application!

Getting Support​

You can get direct access to the Engineering and Product team by joining our Discord Community or email us at [email protected].

Common errors​

Process function limitation​

Only one .process function is supported per Turbine data app.

go mod init for a new Golang Turbine Data Application​

You'll see a message like below when initializing a new Golang Turbine Data Application.

$ meroxa apps init --lang go --path ~/ my-data-app
    ✔ Application directory created!
    ⚡ /home/me/my-data-app is not under $GOPATH/src; skipping go module initialization
    For guidance, visit https://docs.meroxa.com/overview#common-errors
    ✔ Git initialized successfully!
Turbine Data Application successfully initialized!
You can start interacting with Meroxa in your app located at "/home/me/my-data-app".
Your Application will not be visible in the Meroxa Dashboard until after deployment.

At this point, your application's directory has code in it, but the Golang module is not initialized and the dependencies are not set up.

Background​

Let's rewind. When developing with Golang, it has certain expectations of your development environment.

Golang expects a particular directory structure for Golang development environment, aka "Workspaces", particularly for locating private dependencies.

GOPATH indicates where source code and modules should be found.

Depending on how you want to build Golang binaries, you may want to set GOROOT

Setting GO111MODULE, GOPROXY, GONOPROXY, and GOPRIVATE environment variables indicates your preferences for module management.

Remedies​

Turbine skips steps that are likely to fail in order to complete as many steps of the application initialization as possible. You may need to run these skipped steps manually, depending on which one of the following remedies you choose.

If you do not use one of these remedies, or if you use a directory immediately under $GOPATH, you will encounter this error from Golang:

go: cannot determine module path for source directory /home/me/my-data-app (outside GOPATH, module path must be specified)

Remedy 1) Initialize a new application under $GOPATH/src, or better yet under $GOPATH/src/github.com/my-org

If you haven't made any code changes to your application, this is the easiest option. You won't need to run any manual steps in the new directory in this case.

$ meroxa apps init --lang go --path $GOPATH/src/github.com/my-org my-data-app

Remedy 2) Move your new application directory under at least $GOPATH/src, or better yet $GOPATH/src/github.com/my-org

$ mv ~/my-data-app $GOPATH/src/github.com/my-org

Remedy 3) Keep the code where it is, but create a logical link under $GOPATH/src

$ ln -sf ~/my-data-app $GOPATH/src/github.com/my-org/my-data-app

Finally, manually run the steps that were skipped.

$ go mod init
$ go get github.com/meroxa/turbine-go
$ go get github.com/meroxa/turbine-go/runner
# And either
$ go mod download
# or
$ go mod vendor

An example of what that output will look like:

[email protected]:~/go/src/github.com/my-org/my-data-app  $ git status
On branch main
nothing to commit, working tree clean

[email protected]:~/go/src/github.com/my-org/my-data-app  $ go mod init
go: creating new go.mod: module my-data-app
go: to add module requirements and sums:
    go mod tidy

[email protected]:~/go/src/github.com/my-org/my-data-app  $ go get github.com/meroxa/turbine-go
go: added github.com/meroxa/turbine-go v0.0.0-20220712011240-51913fd92155
go: added github.com/tidwall/gjson v1.14.1
go: added github.com/tidwall/match v1.1.1
go: added github.com/tidwall/pretty v1.2.0
go: added github.com/tidwall/sjson v1.2.4
[email protected]:~/go/src/github.com/my-org/my-data-app  $ go get github.com/meroxa/turbine-go/runner
[email protected]:~/go/src/github.com/my-org/my-data-app  $ go mod download
[email protected]:~/go/src/github.com/my-org/my-data-app  $ git status
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    go.mod
    go.sum

Happy coding!

Uncommitted changes​

You must commit to your local project git repository before deploying. If you have any uncommitted changes, you will encounter the following error:

$ meroxa apps deploy
Checking for uncommitted changes...
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   index.js

no changes added to commit (use "git add" and/or "git commit -a")
Error: unable to proceed with deployment because of uncommitted changes

To resolve this error, simply follow the instructions by running the following commands:

$ git add .
$ git commit -m "First commit"
[master (root-commit) a1b2c34] First commit
10 files changed, 953 insertions(+)
// List of Turbine data app project files

Once you've committed, you can run the meroxa apps deploy command.

Could not find resource​

Your production data is available to your Turbine data applications through resources configured on the Meroxa Platform. Any references to resources must correspond to a resource that exists. If the resources do not exist or are misidentified in your code, you will run into the following error:

$ meroxa apps deploy
Checking for uncommitted changes...
    ✔ No uncommitted changes!
Validating branch...
    ✔ Deployment allowed from master branch!
Preparing application "liveapp" (language) for deployment...
    ✔ Application built!
    x Resource availability check failed
Error: could not find resource "my-resource-with-a-typo";

 ⚠️  Run 'meroxa resources list' to verify that the resource names defined in your Turbine app are identical to the resources you have created on the Meroxa Platform before deploying again

If you've already configured your resources, check your code to make sure you've identified your resources appropriately.

If need be, refer to our documentation on creating resources to configure your resources.

Once the discrepancy has been identified, you can run the meroxa apps deploy command again.

Collection validation errors​

Downstream collection used in another app​

All destination resource collections referenced in your Turbine code are checked against data app instances running on the Meroxa Platform when deployed. If another data app uses the same destination resource collection, the deployment process will fail and result in an error. This validation prevents accidental record duplication in downstream resources.

$ meroxa apps deploy
Checking for uncommitted changes...
    ✔ No uncommitted changes!
    ✔ Feature branch (test) detected, setting app name to liveapp-test...
Preparing application "liveapp-test" (javascript) for deployment...
    ✔ Application built!
    x Resource availability check failed
Error: ⚠️ Application resource "prod" with collection "orders" cannot be used as a destination. It is also being used as a destination by another application "liveapp".

Please modify your Turbine data application code. Then run `meroxa apps deploy` again. To skip collection validation, run `meroxa apps deploy --skip-collection-validation`.

Source and destination collection are the same in the application​

If a data app references a source resource collection that is the same as the destination resource collection in the Turbine code, this will result in the deploy process failing with a resulting error. This validation prevents accidental looping effects within a data app.

$ meroxa apps deploy
Checking for uncommitted changes...
    ✔ No uncommitted changes!
    ✔ Feature branch (test) detected, setting app name to liveapp-test...
Preparing application "liveapp-test" (javascript) for deployment...
    ✔ Application built!
    x Resource availability check failed
Error: ⚠️ Application resource "prod" with collection "orders" cannot be used as a destination. It is also the source.

Please modify your Turbine data application code. Then run `meroxa apps deploy` again. To skip collection validation, run `meroxa apps deploy --skip-collection-validation`.

Remedy for Collection Validation Errors​

If the changes are correct and you wish to proceed and skip collection validation, run meroxa apps deploy --skip-collection-validation.

Updated Meroxa CLI and Outdated Turbine library​

The Meroxa CLI does not yet track a required version of the associated Turbine libraries. Therefore, you may encounter the situation in which you have just updated to the latest version of the Meroxa CLI and the next time you try to deploy a Turbine Application, you see an error like this:

        Error: flag provided but not defined: -gitsha
        Usage of /apps/security/securityculcxm3352tqoqjmq1lt:
          -deploy
                deploy the data app
          -imagename string
                image name of function image
          -listfunctions
                list available functions
          -listresources
                list currently used resources
          -serve string
                serve function via gRPC

In this case, updating your Turbine library to the latest version should correct the mismatch and unblock deployment.

  • Golang
$ cd <path-to-app>
$ go get -u github.com/meroxa/turbine-go
$ go mod tidy
# optional: save modules to vendor directory
$ go mod vendor

# prepare for next deployment
$ git add .
$ git commit -m "update turbine-go"
  • Javascript
yarn upgrade turbine-js
  • Python
$ pip install -u turbine-py

At this, deploying should succeed again.