A Cloud Model Storage/Repository: The modelix model-server
|
API doc | Repository | buildfile | Artifacts: Nexus GitHub Packages |
The model-server component provides a storage for models.
Bringing models to the cloud requires an alternative to the file system (and git) based storage of models.
Modelix implements a data structure that allows replication between all MPS instances connected to the same model-server.
It is very similar to the git storage format, but instead of files it stores nodes in its key-value data store.
Real-Time Collaboration and Operational Transformation
To support real-time collaboration, in addition to the snapshot of a version, the model-server also stores the operations that were applied on the previous version to produce the new version.
Conflicts between concurrent modification are resolved using operational transformation (OT).
The usually hard part of OT is to guarantee convergence, meaning that all clients end up with the same result independent of the order in which they receive changes.
This problem becomes trivial, because versions are identified by a hash over all the content.
Any divergence will result in a different hash which can easily be detected.
The remaining problem is what to do when a divergence is detected. From git we know that switching between versions is fast, because it only has to apply the difference between these versions. The same is true here: Fixing a divergence is as easy as switching to the correct version. If a client does not have any local changes it does not even need to apply the change operations itself - it can just switch to the new version. So in the rare case of a divergence fixing it is nothing special and is an operation that happens all the time.
MPS and the model-server
While this component is independent of MPS, it is possible to replicates the data of the MPS internal module repository into a running model-server.
This can be achieved using the modelix MPS plugin: One can connect an MPS instances to a model-server, upload the current module repository, and keep it synchronized.
This can also be a local MPS instance without a web editor - both options are supported at the same time.
APIs
Valid keys are keys starting with the PROTECTED_PREFIX ($$$).
For details please refer to the OpenAPI documentation under doc/model-server.json.
JSON web tokens (JWT) based authorization
By default, the model server doesn’t check any permissions. To enable it you can specify the following environment variables.
| Variable | Description |
|---|---|
MODELIX_PERMISSION_CHECKS_ENABLED |
By default, permission checking is enabled when an algorithm for the JWT signature is configured.
This variable can be set explicitly to |
MODELIX_GENERATE_FAKE_JWT |
By default, if no signature algorithm is configured,
a token is generated for all requests with the identity |
MODELIX_JWT_SIGNATURE_HMAC512_KEY MODELIX_JWT_SIGNATURE_HMAC384_KEY MODELIX_JWT_SIGNATURE_HMAC256_KEY |
If JWT tokens are signed with the HMAC algorithm, this variable specifies the secret key. The |
MODELIX_JWT_SIGNATURE_HMAC512_KEY_FILE MODELIX_JWT_SIGNATURE_HMAC384_KEY_FILE MODELIX_JWT_SIGNATURE_HMAC256_KEY_FILE |
Instead of specifying the key directly via |
MODELIX_JWK_URI |
If keys are created and signed by some OpenID connect server the public keys are provided via HTTP. Here you can specify the URI of the key set. Only RSA (256, 284 and 512) keys are currently supported. |
KEYCLOAK_BASE_URL KEYCLOAK_REALM |
Legacy variables for the keycloak based authorization used by the Modelix Helm charts for workspaces. |
The permissions claim of the token is expected to list directly granted permission.
Additional granted permissions may be derived from that list.
For example, an admin permission on a repository will grant write permission on all branches.
A detailed list of all permissions can be found on the model server at /permissions.