# Starcounter Documentation

This is the official documentation for Starcounter. The main parts are:

1. [Starcounter](/2.3.1/starcounter): a short description of what Starcounter is
2. [Tutorial](/2.3.1/tutorial): an introduction to building Starcounter applications
3. [Guides](/2.3.1/guides): explains how different Starcounter features work.
4. [Cookbook](/2.3.1/cookbook): common solutions and patterns in Starcounter

The drop down on the left sidebar lets you choose different versions of the documentation.

Sample apps are available in the [StarcounterApps](https://github.com/Starcounterapps) repository. 


# Getting Started

To get started building apps with Starcounter, follow these steps and you'll know everything you need to build world class web apps.

## 1. Install Visual Studio

Start by installing Visual Studio to use as a development environment. It's not strictly necessary, but it makes everything simpler.

If you don't already have Visual Studio 2015 or 2017 installed, install it from [visualstudio.com](https://www.visualstudio.com/downloads/). The community edition is completely free to download.

## 2. Install Starcounter

When you have the development environment set up, you can download Starcounter. When using Starcounter for the first time, we recommend using either the latest release or release candidate. The latest release is likely more stable but has fewer features than the release candidate. If you don't want to make a choice, click [this link](http://downloads.starcounter.com/download/Starcounter/RC/latest) and you'll get the latest release candidate.

Take a look at the [download page](https://starcounter.io/download/) for system requirements. All versions of Starcounter can be found on [downloads.starcounter.com/downloads](http://downloads.starcounter.com/download).

The installer will guide you through the process and install the Visual Studio Starcounter extension.

## 3. Build and Run an App

To run your first app, open Visual Studio and choose `File -> New -> Project...`. Create a new Starcounter application by going to `Installed -> Templates -> Visual C# -> Starcounter -> Starcounter Application`. If we call it "HelloStarcounter", it will create a project containing a `Program.cs` file that looks like this:

```csharp
using System;
using Starcounter;

namespace HelloStarcounter
{
    class Program
    {
        static void Main()
        {

        }
    }
}
```

For a first simple application, we'll define a `GET` handler that returns an HTML element:

```csharp
using System;
using Starcounter;

namespace HelloStarcounter
{
    class Program
    {
        static void Main()
        {
            Handle.GET("/Hello", () =>
            {
                return "<h1>Hello Starcounter!</h1>";
            });
        }
    }
}
```

Run the application by pressing `F5` and go to `http://localhost:8080/Hello`.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5164fb0-8680-11e7-9944-1f85270462c6%2FHelloStarcounter.PNG?generation=1503327413604872\&alt=media)

If you see the screen above, then you've done everything right so far.

## 4. Do the HelloWorld Tutorial

With everthing set up, you can start the [HelloWorld tutorial](/2.3.1/tutorial). It will teach you the basic features of Starcounter and how to build a standalone app.

## 5. Read More About Starcounter

To get a deeper understanding of Starcounter, read the [Starcounter](https://app.gitbook.com/Starcounter/README.md) section where we describe some of the fundamental concepts. You can then head over to the [topic guides](/2.3.1/guides) and get an in depth look on specific topics.

## 6. Try the Sample Apps

We've developed a bunch of apps that are available for everyone on [GitHub](https://github.com/starcounterapps). Feel free to run them, change them, make issues, and make pull requests. They should also give you an idea of apps are built to interact with each other.

## 7. Build Your Own App

With this, it's time to build your own app. When building, don't hesitate to ask questions in the [Home](https://github.com/Starcounter/Home/issues) repo on GitHub. We are always happy to get feedback on what we can improve.

We hope you'll enjoy Starcounter as much as we do!


# Starcounter

Starcounter is an In-memory Application Platform with a built in [REST](http://en.wikipedia.org/wiki/Representational_state_transfer) style server. It supports [SQL](http://en.wikipedia.org/wiki/Sql) queries and is fully [ACID](http://en.wikipedia.org/wiki/ACID) compliant. Disks secure the transaction log as well as a checkpointed database image.

Starcounter is the platform for building modern applications such as web applications, mobile services and massive real-time software such as games, advertising, and telecom applications.

Starcounter is highly optimized for stellar throughput, concurrency, and low latency. It also features tight integration with web standards such as HTTP, REST, JSON.

Starcounter offers unmatched simplicity to create web browser view-models with zero glue code.


# Collapsing the Stack

**Time to work smarter, not harder. Find out why making the switch to simplified system architecture is the key to beating slow software and maximizing scalability.**

A software vendor’s primary objectives are to attract new users, adapt to the expectations of existing users, and prevent user outflow — all critical to maintaining the agility, competitiveness, and profitability of a business.

Every time the user base grows, the developer is put under pressure — tasked with scaling the entire user platform to accommodate increased workloads, while keeping the UX response time within a fraction of a second. No easy feat considering the development stacks commonly used today, which present significant [trade-offs between system complexity and scalability](/2.3.1/starcounter/collapsing-the-stack/complexity-and-scalability-tradeoff).

This imbalance can be resolved through the adoption of an emerging kind of software technology which takes a [slimmed down approach](/2.3.1/starcounter/collapsing-the-stack/the-future-of-micro-services) to the software stack — allowing developers to work smarter, not harder. The [resulting benefits](/2.3.1/starcounter/collapsing-the-stack/10-benefits-of-collapsing-the-stack) include agility, competitiveness, and a reduced total cost of ownership.


# Complexity and Scalability Tradeoff

In the context of today’s conventional development platforms, the performance capacity of a single server runs out fast. Scaling operations often means scaling out: adding more machines — more layers of complexity — to a multi-tiered network of various kinds of servers, and performance enhancers like multi-node data redundancy, caches, and data grids. An increasing user base leads to a corresponding expansion of tiers and number of machines.

**This scale-out approach comes with some weighty drawbacks:**\
– Higher costs for hardware, software, and maintenance\
– Increased complexity of development and system control\
– Higher disaster risks due to more potential points of failure\
– Greater risk for bottlenecks and implementation bugs\
– Difficulty identifying roots of problems and their posterior fixings\
– The need for costly and mistake-prone inter-module integration and configuration

Together, these drawbacks lead to a drop in performance, reliability, and data consistency — issues impossible to solve by just adding more hardware. This in turn has a negative effect on time-to-market for new releases, user satisfaction, and, as a result, the vitality of the business.

Even if a scaled-out solution contained no complex logic or heavy computations, the amount of hardware required to run it with affordable UX response time is prone to becoming ridiculously large. Unfortunately, such state of affairs reveals not the immaturity of the system’s implementers (as one might reasonably suggest), but rather a considerable fault in the approach itself.

Multi-tier, scale-out, data-excessive architectures can be seen as a tool to solve an effect in popular cases, but not a cause. They work well in different circumstances like social networks, historical data storage, or overnight business intelligence, but produce failures in applications involving the management of valuable resources, including ERP and line-of-business apps.

### Collapsing the Stack

Luckily, instead of attempting to solve the effect—leveraging for local gains with multiple separate tiers—its possible to address the root cause, the development platform itself. Implementing the platform according to updated fundamentals allows for a new breed of applications, globally optimized by simplicity, performance, and modularity. This vision is summarized by the concept of collapsing the stack. For software development, this means making the code concise, eliminating glue code, simplifying system architecture, increasing flexibility, and thus the scalability of the resulting solution. In business terms, this translates to improved agility, competitiveness, and reduced total cost of ownership.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe553f4f0-8680-11e7-9944-1f85270462c6%2Fcollapsed-stack.png?generation=1503327412403955\&alt=media)

Figure 1: Collapsing the software stack; traditional vs. collapsed

Implementing “the collapsed stack” entails making a shift from engineering the tiers and their integration, to operating within a focused application platform. All features from tiers (like network communication, data persistency, and failover) are available as before. But now the tiers are either virtualized or removed in favor of simpler facilities. The shift does not mean moving from the modularity of layers to a monolithic product, however. The difference is that highly modular solutions can be facilitated without sacrificing performance, simplicity, and cost.


# The Future of Micro-Services

In light of the recent micro-services movement, the inception of a new breed of in-memory technology is a central component.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe525e010-8680-11e7-9944-1f85270462c6%2Fstarcounter_microservices.png?generation=1503327411540469\&alt=media)

Figure 2: Evolution of the database; Monolithic, micro-services, and Starcounter’s micro-apps

The currently prevailing multi-tier micro-services platform sees the application and database as two disjoint entities, their communication of data dependent upon the constant exchange of signals. But more points of connection equals more points of system vulnerability. Laws of physics define a strict upper limit to the performance of such a multi-tier architecture: It takes 8 minutes for light to travel from Sun to Earth, 130ms for a signal to get from Australia to the U.S. by wires. Even if these databases and applications were to run on the same machine, they would still communicate and thus signal via a silicon chip. The solution in this context of this architecture is to minimize signaling by placing the separated entities as close together as possible.

First-generation in-memory databases made a significant shift from operating data on disks to operating in-memory by securing log to disk, which is multiple orders of magnitude faster. Software platforms like Starcounter’s, which employ a collapsed stack architecture, demonstrate yet another leap ahead—shrinking database and application tiers into a single layer. Such a collapsed stack architecture offers a high-performing, open-ended node for running a set of upload-able micro-apps. The data integration, which is considered challenging for micro-services, can be solved for micro-apps by way of efficient in-memory data sharing.

Whereas today’s storage procedures will often put chunks of code closer to the database to increase performance, with the collapsed stack approach it is no longer necessary to dissipate logic between database and app code since all applications running in a platform operate physically the same data instances that the database owns. Thus, delivery of data from the database to the app is not needed. This translates to millions of fully ACID transactions per second on a modest server.

What’s more, by shrinking the [database and application tiers into a single layer](/2.3.1/starcounter/integrated-database-and-web-server), additional value can be realized via the elimination of messages relayed between the web and app server, inter-process communication, data redundancy, and similar “middlemen”. In addition, the glue code which was binding the layers goes away, resulting in the pure, beautiful business logic expressed by concise code.

### Who will Benefit Most from Collapsing the Stack?

Any data or performance-critical business is a strong candidate for collapsing the stack and adopting an in-memory platform. Likewise, any business demanding agility and performance is a good match. Such a platform can be used within any vertical, but likely within industries including banking, finance, retail, internet, telecom and gambling/gaming.

An excellent usage example, documented by the International Data Corporation (IDC), is represented by Swedish retailer Gekås Ullared, who was driven to make the switch to Starcounter’s in-memory application platform at a critical time when their accelerated growth was overwhelming their databases— and their IT budget.

In Starcounter Gekås found a system architecture equipped to keep up with their scaling data volumes and changeable application requirements, without necessitating a corresponding increase in workforce.

According to the IDC study results, “Gekås is still growing, but as the business growth ramps up and the database expands to keep up, performance remains a constant. Today, the company has two Starcounter databases; one to manage logistics, and the other for in-store management, including the point-of-sale terminals (POSTs). All these systems, including system, application, and database maintenance, are handled by a staff of seven people.”


# 10 Benefits of Collapsing the Stack

The best way we can illustrate the benefits of the collapsed-stack approach is by sharing some of the results we’ve achieved with our own in-memory database and application platform:

### 1. Fast, responsive and multi-platform GUI

A rich web-based GUI running on multiple desktop and mobile platforms offers a faster, slicker, simpler, more stable and responsive user experience. As an outcome, new users adapt to software more rapidly, and with greater pleasure.

### 2. Improved agility for business

Good modularity, compose-ability, and extensibility reduces time from production to customer hands, and enables instant response to emerging customer needs. With an in-memory computing platform, solution developers and consumers don’t need to make trade-offs between performance, modularity, and reliability— and instead, can build a solution with strong values for all of these points.

### 3. Faster deployment cycle and instant module/app integration

A data integration feature enables development of independent modules/apps without the need for app-to-app integration or universal bus and master data management orchestration. Instead, data and functionality are integrated organically (and with a minimal amount of code) on the level of the in-memory platform and user interface, allowing for their efficient collaboration.

### 4. Better technology learning curve

Thanks to technology enabled for multiple popular platforms (like .NET, Node.JS, Java, Windows/Linux/OSX), newcomer developers are ready to go in 1 or 2 weeks, with the ability to create new functionalities and modify existing modules. Development of integration with existing modules is not needed and module/application development becomes 100% independent.

### 5. Reduced development and maintenance complexity, significantly lower implementation risks.

An empowered, declarative programming style and the elimination of unnecessary APIs produces fewer lines of code and bugs. A short-term investment in studying Starcounter technology transforms into the long-term benefit of a predictable development cycle.

### 6. Strong security and data integrity guarantees

Starcounter’s in-memory technology enables strong ACID properties, which are strictly required for business transactions, while a server-centric architecture with a thin web-based client provides high security guarantees. Technology is designed to eliminate opportunities for unauthorized access penetration, query injection, and other popular vulnerabilities met in todays ERP systems. The data binding model exposes only a mirrored (server-secured) UI representation to let the server consistently verify the input. Such properties are especially important in resource-sensitive domains such as trading, banking, and accounting.

### 7. 100x less hardware, 100x more performance

By empowering in-memory computing technology, hardware and software costs are reduced up to 100 times. Customers can run their solution on 1 server instead of 100, which reduces costs for hardware purchasing and maintenance, cluster administration staff (system administrators, DBAs), software licenses to run the cluster, and reduces power consumption closer to nature-friendly levels. At the same time, back-end performance is increasing proportionally, offering mixed OLTP/OLAP workloads and responsive UX, unavailable with traditional data processing and integration approaches (RDBMS, MDM workflow, independent APIs, and similar).

### 8. Improved reliability

Starcounter’s customer-proven in-memory database combined with the elimination of server fleets substantially reduces the risk of operational disaster (like data loss due to hardware malfunction, or software bugs), and a consecutive painful recovery. If a sever’s recovery is required due to power loss, it takes seconds to minutes for snapshot and log recovery, compared with the hours required for known conventional DBMS-centric solutions.

### 9. Improved data ownership

Sharing data allows it to be managed in a meaningful way, and empowers strong normalization coupled with multi-faceted data representation without redundancy and logic dissipation over modules. In addition, this eliminates the need for customized master data management solutions and services (like establishing master data, performing data cleansing, and de-duplication).

### 10. Rich integration capabilities, and exemption from vendor lock-in

Enterprise system users will benefit from running highly modular solutions built with a full set of integrated modules, as requested by the business. Wrapper apps functioning inside the platform will allow for the seamless integration of legacy systems into the new solution, providing enterprises with a smooth migration path, as well as an escape from vendor lock-in situations.

### Are you Ready for the Future?

The approaching Internet of Things era makes utilization of in-memory platforms and collapsed stacks unavoidable. To put things in perspective, moving from around 7 billion humans to 60 billion humans—users and devices— means we can expect an increase of at least 7 times more transaction loads. As soon as the year 2020, 26 to 50 billion devices are expected to be connected to the network.

While device interaction will be seamless for users, it shall, in fact, be an orchestration of device-driven transactions conducted via net-based apps operating on data. Device chains, federations, multi-party communications and the like—all these are IoT phenomena with added complexity to be considered. And the whole turns out to be greater than the sum of the parts.

All the above practically leads to an astonishing increase in transactional workload which API-based web services have to meet. If thousands of user connections per second might become a problem for a REST-service today, consider the level of impedance when it comes to responding to millions of devices per second.

Now is the time to collapse the stack and get up to speed with the future. Luckily, in-memory technologies have arrived in perfect time.


# Integrated Database and Web Server

Let's deepen into technical details in order to understand how two layers, application and database, are physically merged into one.\
We can say that database and application server are represented by two parallel running processes. In the picture they are presented as ScDATA and ScCODE respectively. Inbound and outbound traffic towards those processes is initially handled by Gateway process.

![](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-KtlXIfH9-aKypOu9PcB%2F-KtlXLIx1NBxEuqS_zbR%2FConnects.gif?alt=media\&token=d9ecfe43-57b9-401b-b63b-ede9027a43f3)

Distinction of depicted processes:

1. **ScNETWORKGATEWAY** represents a Gateway for network traffic, delivering requests to execute and sending responses back (read more [here](/2.3.1/guides/network/network-gateway));
2. **ScCODE** represents Codehost as a run-space for all the applications operating on the same Database;
3. **ScDATA** represents Database and manages database memory handling for Codehost activity.

Communication between processes is organized through the shared memory. This allows processes to efficiently exchange messages, preserving processes isolation, security and consistency.

## Codehost Isolation

Gateway and Codehost processes operate on one shared memory segment, while Database and Codehost on another, meaning that Gateway has no direct access to Database process - it can only communicate with the Codehost.

This is done for multiple reasons:

* Failure/restart of the Codehost process does not affect the Database.
* Developers can iterate application versions and update those without Database process restart (only Codehost is restarted and being reconnected to Database);
* Database is isolated from networking process therefore it is impossible to affect the Database directly through Gateway.
* In the future, each application will have their own Codehost processes to ensure app independence and overall system stability.


# Hello World - Tutorial

In this "Hello World" tutorial you will learn how to use the essential features of Starcounter by building an expense tracker complete with a database, interactive UI and real-time syncing.

We will use C# as the server-side language. All the app logic will be written in C# and execute in the database. The client-server communication will be handled by [Palindrom](/2.3.1/guides/web-apps/palindrom) over HTTP and WebSocket. This is how we recommend building Starcounter apps.

## Content

In the first two steps, you will create the simplest possible application that binds the database to the view.

In the four following steps, you will create an expense tracker by establishing a UI and building functionality that allows CRUD operations on the database.

## Final Result

![Final result](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe504c380-8680-11e7-9944-1f85270462c6%2Fresizedpart6.gif?generation=1503327411295174\&alt=media)

## Requirements

To get through this tutorial, you will need to fulfill the [system requirements](http://starcounter.io/download/) which includes having Visual Studio 2015 or 2017 (not VS Code) and the [latest Release Candidate](http://downloads.starcounter.com/download) of Starcounter installed.

## Extra Resources

Since the author is a mere mortal, there might be mistakes in the code or text. If you experience that something doesn't work as intended, you can look at the [source code](https://github.com/StarcounterApps/HelloWorld). Each commit in this repository represents one step in the tutorial. For further explanations of the concepts discussed, you can take a look at the [guide section](/2.3.1/guides).

Are you ready?

Let's get started!


# Create a Database Class

Welcome to the first part of our Hello World tutorial!

We will start by creating a Starcounter application in Visual Studio by going to `New Project -> Templates -> Visual C# -> Starcounter -> Starcounter Application`. We will name the application `HelloWorld`.

## Add a Persistent Class

Create a new class called `Person` with the attribute [`[Database]`](/2.3.1/guides/database/creating-database-classes) in the `Program.cs` file. This attribute tag will make all instances of the class persistent.

Add the properties `FirstName` and `LastName` to this class. Your code should now look like this:

{% code title="Program.cs" %}

```csharp
using Starcounter;

namespace HelloWorld
{
    [Database]
    public class Person
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
    }

    class Program
    {
        static void Main()
        {

        }
    }
}
```

{% endcode %}

## Add an Instance to the Class

Add a first instance to the class by defining a new person, its properties, and wrapping it in a [`Db.Transact()`](/2.3.1/guides/transactions/using-transactions#dbtransact). Using a transaction allows us to access database objects and makes the changes inside the transaction atomic and isolated.&#x20;

{% content-ref url="/pages/-KtAuTVtC46LosLTlqYu" %}
[Transactions](/2.3.1/guides/transactions)
{% endcontent-ref %}

{% code title="Program.cs" %}

```csharp
class Program
{
    static void Main()
    {
        Db.Transact(() =>
        {
            var person = Db.SQL<Person>("SELECT p FROM Person p")
                .FirstOrDefault();
            if (person == null)
            {
                new Person
                {
                    FirstName = "John",
                    LastName = "Doe"
                };
            }
        });
    }
}
```

{% endcode %}

The if statement here checks if you already have a `Person` in the database by accessing the first result that we get from the query. If that is the case, you do not need to create a new one. Without it, we would create a new instance of `Person` every time we run the program, which we do not intend to do.\
\
Remember to import `System.Linq` for `FirstOrDefault`.

## Result

Start your program with Starcounter by clicking F5 in Visual Studio. To see for yourself, open the administrator at `localhost:8181/#/databases/default/sql` and enter `SELECT * FROM HelloWorld.Person`. This will display all the instances, represented as rows, of the `Person` class. Note that these instances are persistent. You can restart the application, or even the computer, and the instances will still be there.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe54241b0-8680-11e7-9944-1f85270462c6%2Fpart1resized.gif?generation=1503327413876273\&alt=media)

For the next step, we'll add a UI which will help us to display the data in the browser.

If you get any errors, check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/4c91c301444dc074172851df7d4153ad9b5869c1).


# Create a Real Time UI

To create a user interface (UI), we establish a [MVVM](https://en.wikipedia.org/wiki/Model–view–viewmodel) pattern by adding a [view-model](/2.3.1/guides/web-apps/starcounter-mvvm#tier-2---the-view-model) and a corresponding [view](/2.3.1/guides/web-apps/html-views). The view-model will act as a representation of the view and as a midpoint between the view and the database.

## Setup the File Structure

Let's create the file structure to contain a view and a view-model. Follow these steps to create the structure:

1. In the Solution Explorer in Visual Studio, add a folder with the name `wwwroot` in the `HelloWorld` project.
2. Inside this folder, create another folder with the name `HelloWorld`, the same name as the project.
3. Add an HTML file into this folder by right-clicking and then choosing `Add -> New Item... -> Starcounter -> Starcounter HTML template with dom-bind`. Name this file `PersonJson.html`.
4. Add a JSON file, which will be the view-model, into the root of the project by going to `Add -> New Item... -> Starcounter -> Starcounter Typed JSON with Code-behind`. By doing this, you create a `json` and `json.cs` file. Both of these should have the name PersonJson.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe58fec80-8680-11e7-9944-1f85270462c6%2Ffile-structure.PNG?generation=1503327413356112\&alt=media)

With a solid file structure, we can continue by creating the view-model.

## Define the View-Model

In the JSON file, create three properties called `Html`, `FirstName`, and `LastName`. The values of the properties `FirstName` and `LastName` will be bound to the view *and* the database. Because of that, it is crucial, for this example, that you name these properties the same as the properties that we have in the `Person` class. That will allow Starcounter to recognize that the `Person` object and the `PersonJson` view-model represents the same things and bind the values of their properties. Learn more about this in the [docs](/2.3.1/guides/typed-json/json-data-bindings).

The value of the `Html` property is the path to the view that the view-model should be connected to.

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName": "",
  "LastName": ""
}
```

{% endcode %}

## Create the HTTP Handler

To get the view-model and the corresponding view to the client, an HTTP handler has to be created. This handler sets the specific database object that the view-model should be bound to, creates a [session](/2.3.1/guides/web-apps/sessions), and returns the view-model to the client.

This handler will only return the JSON tree that we defined earlier and not any HTML, which is what we want to render. The solution is to use [middleware](/2.3.1/guides/network/middleware). For this app, and most other Starcounter apps, we use the `HtmlFromJsonProvider` and `PartialToStandaloneHtmlProvider` middleware. These affect the pipeline by catching the outgoing JSON, finding the HTML at the `Html` path, wrapping the HTML to form a complete HTML document, and forwarding it to the client. By doing this, a complete HTML document can be sent to the client, even if the only thing that's returned from the handler is a simple JSON tree.

This is how the handler and middleware looks in code:

{% code title="Program.cs" %}

```csharp
Application.Current.Use(new HtmlFromJsonProvider());
Application.Current.Use(new PartialToStandaloneHtmlProvider());

Handle.GET("/HelloWorld", () =>
{
    Session.Ensure();
    var person = Db.SQL<Person>("SELECT p FROM Person p")
        .FirstOrDefault();
    return new PersonJson { Data = person };
});
```

{% endcode %}

## Define the View

The view will initially be a simple interface displaying the `FirstName` and `LastName` of a `Person` instance.

Since the middleware wraps the HTML to create a complete HTML document, we only need to define an HTML fragment wrapped in the `template` element.

To create a two-way binding between the view and the view model, we choose to use Polymer. There are three parts needed to make the bindings work:\
1\. Import Polymer\
2\. Use `<template is="dom-bind">` to allow the use of bindings in a fragment\
3\. Bind the specific properties using the double bracket syntax like this: `{{model.FirstName}}`. `model` represents the JSON file, so `{{model.FirstName}}` is the `FirstName` value in `PersonJson.json`.

This is how it looks in code:

{% code title="PersonJson.html" %}

```markup
<link rel="import" href="/sys/polymer/polymer.html"/>
<template>
    <template is="dom-bind">
        <fieldset>
            <label>First name:</label>
            <strong>{{model.FirstName}}</strong>
        </fieldset>

        <fieldset>
            <label>Last name:</label>
            <strong>{{model.LastName}}</strong>
        </fieldset>
    </template>
</template>
```

{% endcode %}

## Result

Now, we have established a real-time model-view-view-model (MVVM) binding. The JSON, which is our view-model, is bound to the model (database) with no latency; our view, the HTML, is in turn bound to the JSON, which is synced in real time using WebSocket and HTTP. Polymer helps us display this instantaneously to the user.

Check out how it looks by starting the application with F5 and go to `http://localhost:8080/HelloWorld` in your web browser.<br>

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5903aa0-8680-11e7-9944-1f85270462c6%2Fpart2.png?generation=1503327412075029\&alt=media)

It's impossible for us to see the immediate changes as there is no way for the user to change the info. Let us fix that by adding some interactivity!

If you get any errors, you can check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/ce3e787313aacbd6d8f6d18956ab39e24befc452).


# First Interactive UI

The initial UI will allow us to change the name of a person and save those changes to the database. This requires a couple of things:\
1\. The database should be able to commit changes to the database at any time\
2\. The view-model needs to be writable from the view\
3\. The view has to allow input and bind the input to the view-model

## Creating a Long-Running Transaction

To be able to commit changes to the database at any time, a [long running transaction](/2.3.1/guides/transactions/long-running-transactions) is needed. This transaction should be attached to the `PersonJson` view-model.

In code, that is done by wrapping everything in our handler inside a `Db.Scope`:

{% code title="Program.cs" %}

```csharp
Handle.GET("/HelloWorld", () =>
{
    return Db.Scope(() =>
    {
        Session.Ensure();
        var person = Db.SQL<Person>("SELECT p FROM Person p")
            .FirstOrDefault();
        return new PersonJson { Data = person };
    });
});
```

{% endcode %}

## Writable View-Model

To make properties in the view-model writable from the view, a dollar sign it added to the end of it. With this, `"FirstName"` becomes `"FirstName$"` and `"LastName"` becomes `"LastName$"`. The view-model should then look like this:

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": ""
}
```

{% endcode %}

## Trigger Property

As mentioned earlier, we also want the possibility to save at will. In order to do this, there needs to be some kind of communication between the view and the [code-behind](/2.3.1/guides/typed-json/code-behind). This can be accomplished using a trigger property which is basically an integer that can be changed from the client and handled in the code-behind.

This is how it should look:

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": "",
  "SaveTrigger$": 0
}
```

{% endcode %}

## Handling Changes in the Code-Behind

To act on the change in the view-model that is triggered from the view, an event handler can be registered in the code-behind.

In this case, where the goal is to save, the following code can be used:

{% code title="PersonJson.json.cs" %}

```csharp
using Starcounter;

namespace HelloWorld
{
    partial class PersonJson : Json
    {
        void Handle(Input.SaveTrigger action)
        {
            Transaction.Commit();
        }
    }
}
```

{% endcode %}

`Input.SaveTrigger action` makes the method run when a change is detected in the `SaveTrigger` value. Note that we do not need to use a `$` here like in the view-model. The rule is that we use `$` for the view, and view-model, but not in the application code.

`Transaction.Commit()` commits the current state of the view-model to the database so that the data is accessible from other transactions.

With server-side view-models like this, you don't have to write a single line of "glue code" to update the view in HTML. Any change in the view-model made in C# will instantly be synced to the client using [Palindrom](/2.3.1/guides/web-apps/palindrom), which in turn automatically renders because of Polymer's data bindings. This saves you from creating single-purpose REST APIs, need for double validation of user input, and more. This also means that all logic that can be written on the server-side should be written on the server-side to enjoy these benefits.&#x20;

### Creating Interactive Elements

Now, with a view-model that is writable and a database which allows commits at any point in time, the view can include elements that change the properties in the view-model.

We'll change our previous text elements to input elements and add a button:

{% code title="PersonJson.html" %}

```markup
<template>
    <template is="dom-bind">
        <fieldset>
            <label>First name:</label>
            <input value="{{model.FirstName$::input}}">
        </fieldset>

        <fieldset>
            <label>Last name:</label>
            <input value="{{model.LastName$::input}}">
        </fieldset>

        <button value="{{model.SaveTrigger$::click}}" onmousedown="++this.value">Save</button>
    </template>
</template>
```

{% endcode %}

The `::input` declaration on the input value sets up an event listener. It updates the property it's bound to on every keystroke. This means that every time a change is made in the input field, the view-model will reflect that change.

To increment the `SaveTrigger$` value in the view-model, we bind it to the value on the button and attach a `::click` event listener. We then increment this value when the button is pressed.

## Result

We now have a program where we can change the view-model in real time and then commit our changes to the database at will. To see how it looks, start the application with F5 and go to `http://localhost:8080/HelloWorld` in the browser. You should see two input boxes with their respective label and a button below.

If you are an especially curious person, you can try to change the name and then take a look at the database again with SQL. Here's how it should work:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe4e97350-8680-11e7-9944-1f85270462c6%2Fpage3resized.gif?generation=1503327410981669\&alt=media)

Neat! Right? The next step is to display the name change in real time and let the code-behind calculate the full name.

If you get any errors, you can check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/e16d15bb6ea3d1253d21496328045e0036af4d5a).


# Computed Properties

Starcounter makes it easy to use computed properties in the data model. Computing values on the fly is often as fast as accessing cached data and brings additional benefits. It allows you to save memory and always be sure that you get the current value.

Let us compute the `FullName` of a person from their `FirstName` and `LastName` and display it with minimal delay.

## Preparing the View-Model

To synchronize the computed property between view and code-behind, simply add the `FullName` property to the view-model.

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": "",
  "SaveTrigger$": 0,
  "FullName": ""
}
```

{% endcode %}

Notice that we don't have to make `FullName` writable because we will modify it from the code-behind and not the view.

## Compute in Code-Behind

There are two ways to implement computed properties, in the code-behind or in the database class. For this tutorial, it'll be done in the code-behind.

The `FullName` property can be calculated by concatenating `FirstName` and `LastName`.

{% code title="PersonJson.json.cs" %}

```csharp
partial class PersonJson : Json
{
    public string FullName => $"{FirstName} {LastName}";

    void Handle(Input.SaveTrigger action)
    {
        Transaction.Commit();
    }
}
```

{% endcode %}

This property will now be bound to the property with the same name in the view-model and always be up to date.

## Display the Computed Property

To display this computed property, we just have to add it to to the view. This is done the same way as earlier; by using a Polymer binding:

{% code title="PersonJson.html" %}

```markup
<template>
    <template is="dom-bind">
        <h1>Hey, {{model.FullName}}!</h1>

        <fieldset>
            <label>First name:</label>
            <input value="{{model.FirstName$::input}}">
        </fieldset>

        <fieldset>
            <label>Last name:</label>
            <input value="{{model.LastName$::input}}">
        </fieldset>

        <button value="{{model.SaveTrigger$::click}}" onmousedown="++this.value">Save</button>
    </template>
</template>
```

{% endcode %}

## Result

Start the application and see how the computed property is calculated keystroke by keystroke and displayed instantly:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5448ba0-8680-11e7-9944-1f85270462c6%2Fpart4resized.gif?generation=1503327412590312\&alt=media)

The next step is to practice working on multiple object instances and relations by turning the app into a simple expense tracker.

If you get any errors, you can check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/69cfcb0bd2dedf268b4d97fcb24cab4da3f40190).


# Expense Tracker

We will now turn our application into a simple expense tracker.

With Starcounter, views and view-models can be broken up into parts and [nested](/2.3.1/guides/web-apps/html-views#using-partials). This allows for increased composability and modularity. In this case, the two main concepts in this app, `Person` and `Expense`, will be built with separate view-models and views and then nested to create a coherent whole. Let's start by creating the appropriate files.

1. Add a new Starcounter "HTML template with dom-bind" in the HelloWorld folder together with `PersonJson.html`. Name it `ExpenseJson.html`.
2. Add a new Starcounter "Typed JSON with Code-behind" file to the HelloWorld project together with `PersonJson.json` and `PersonJson.json.cs`. Name it `ExpenseJson.json`.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe528c640-8680-11e7-9944-1f85270462c6%2FExpense-file-structure.png?generation=1503327412070922\&alt=media)

## Define the Expenses

To define the expenses, three parts are needed:

1. A database class which can store the expenses&#x20;
2. A view that can display the expenses&#x20;
3. A view-model to bind the database class and the view

### Database Class

To keep it as simple as possible, the expense only needs to contain information about its cost or amount, a description of the expense, and a reference to the person that spent the expense.

This is how it looks in code:

{% code title="Program.cs" %}

```csharp
[Database]
public class Expense
{
    public Person Spender { get; set; }
    public string Description { get; set; }
    public decimal Amount { get; set; }
}
```

{% endcode %}

### View-Model

The information that we want to have in the view is the description and the amount, which should be writable from the view since we're building an expense tracker, so that's what we add to the view-model in addition to the HTML path.

This is the result:

{% code title="ExpenseJson.html" %}

```javascript
{
  "Html": "/HelloWorld/ExpenseJson.html",
  "Description$": "",
  "Amount$": 0
}
```

{% endcode %}

### View

The view for the expense can be extremely simple. It will only contain two input fields that are bound to the properties in the view-model:

{% code title="ExpenseJson.html" %}

```markup
<template>
    <template is="dom-bind">
        <input value="{{model.Description$::input}}">
        <input value="{{model.Amount$::input}}">
    </template>
</template>
```

{% endcode %}

## Nest Expenses Within a Person

Now that the concept of an expense is defined, it should be connected to the concept of a person so that a person owns multiple expenses. There are a couple of steps to follow to make that happen.

1. The `Person` class should be linked to its expenses. Right now, each expense is bound to a `Person`. To create a one-to-many relationship, the `Person` should also be linked to the expenses.
2. The view-models of the expenses of a specific `Person` object should be nested into the view-model `PersonJson`. This nesting makes it easy to display the expenses of a person.&#x20;
3. The views corresponding to the nested expense view-models should be stamped out inside the person view.

### Extending the Person Class

To link each `Person` object with its `Expense` objects, we'll utilize the fact that each `Expense` has the property `Spender`. Thus, a SQL query can be made to find all the `Expense` objects where the `Spender` is the current `Person`. In addition to containing a reference to all its `Expense` objects, it might be useful for the `Person` class to contain a propery for the aggregated `Amount` of the `Expense` objects it has. This can also be accomplished with a SQL query.

In code, it looks like this:

{% code title="Program.cs" %}

```csharp
[Database]
public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public IEnumerable<Expense> Expenses => Db.SQL<Expense>(
            "SELECT e FROM Expense e WHERE e.Spender = ?", this);
            
    public decimal CurrentBalance => Db.SQL<Expense>(
            "SELECT e FROM Expense e WHERE e.Spender = ?", this)
            .Sum(e => e.Amount);
}
```

{% endcode %}

### Extending the View-Model

With the properties `Expenses` and `CurrentBalance` in place, we can bind them to the view-model in order to use them in the view.

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": "",
  "SaveTrigger$": 0,
  "FullName": "",
  "Expenses": [{}],
  "CurrentBalance": 0
}
```

{% endcode %}

When binding the view-model with properties of database classes, the types have to be specified. `Expenses` is defined as an array of objects, that is not quite enough. We also need to set what type of object it should contain. This can be done in the code-behind for `PersonJson` by creating a constructor that specifies the type to be `ExpenseJson`. This is how it looks in code:

{% code title="PersonJson.json.cs" %}

```csharp
static PersonJson()
{
    DefaultTemplate.Expenses.ElementType.InstanceType = typeof(ExpenseJson);
}
```

{% endcode %}

### Nesting the View

With the expenses in the view-model, they are easy to include in the view. We simply need to loop over and stamp them out on the page. This can be done with the custom HTML template element `dom-repeat`. It is similar to C# `foreach` loop.

The elements that it should loop over are the expenses. For each expense it should stamp out the view for that particular expense. `imported-template` is a custom element that helps with that by acting as an insertion point.

In addition to this, we want to display the current balance and change the headline to reflect the actual purpose of the page:

{% code title="PersonJson.html" %}

```markup
<template>
    <template is="dom-bind">
        <h1>{{model.FullName}}'s expense list</h1>

        <fieldset>
            <label>First name:</label>
            <input value="{{model.FirstName$::input}}">
        </fieldset>

        <fieldset>
            <label>Last name:</label>
            <input value="{{model.LastName$::input}}">
        </fieldset>

        <button value="{{model.SaveTrigger$::click}}" onmousedown="++this.value">Save</button>

        <hr>

        <template is="dom-repeat" items="{{model.Expenses}}">
            <div>
                <template is="imported-template" model="{{item}}" href$="{{item.Html}}">
                </template>
            </div>
        </template>

        <hr>

        <h2>Current Balance: {{model.CurrentBalance}}</h2>
    </template>
</template>
```

{% endcode %}

## Create New Expenses

If you start the application now, you will not see any expenses because no expenses have been added to the database yet. Implementing the functionality to add new expenses is quite simple. Three things are needed:\
1\. A trigger in the view-model to signal when to add an expense\
2\. A button to activate the trigger\
3\. A handler to add the new expense

### Add Trigger Property

The trigger property will look almost identical to the save trigger:

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": "",
  "SaveTrigger$": 0,
  "FullName": "",
  "Expenses": [{}],
  "NewExpenseTrigger$": 0,
  "CurrentBalance": 0
}
```

{% endcode %}

### Add Button

The button to create new expenses simply have to increment the trigger we just defined. It should look like this:

{% code title="PersonJson.html" %}

```markup
<button value="{{model.NewExpenseTrigger$::click}}" onmousedown="++this.value">Add new expense</button>
```

{% endcode %}

This button should be added below the list of expenses.

### Implement the Handler

To act on the trigger, we'll create a handler in the code-behind. Since the view-model is bound to the `Person` object that holds a reference to a collection of `Expense` objects, we only need to add an `Expense` object to the database and it will immediately be synched to the view-model and placed in the view.

{% code title="PersonJson.json.cs" %}

```csharp
void Handle(Input.NewExpenseTrigger action)
{
    new Expense()
    {
        Spender = this.Data as Person,
        Amount = 1
    };
}
```

{% endcode %}

`This.Data` is the current database object, which in this case is the `Person` that added a new expense.

## Result

Run the program and try to add expenses, change their cost, and see the current balance change in real time.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe528ed50-8680-11e7-9944-1f85270462c6%2FPart5resized.gif?generation=1503327413432899\&alt=media)

With every keystroke, the UI is updated almost instantly from the database. Starcounter's in-memory database makes this possible. There's no delay, everything simply happens at the moment the user interacts with the view.

If you get any errors, you can check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/a6e91b5a2dbedd49d5f228b4fca55487f20c1dda).


# Cancel and Delete

You have probably noticed that in the current app you can only add new expenses. That means that the database and the list of expenses will continue to grow endlessly.

In this part, we will add a cancel and delete button which allows the user to either cancel a change that has not been committed or delete all the expenses of a person.

Similar to how the "Add new expense" button was implemented, the process to add this functionality requires three steps:\
1\. Add trigger properties\
2\. Add elements to increment the trigger properties\
3\. Add event handlers

## Adding Trigger Properties

We start by adding our needed trigger properties for our future buttons to the `PersonJson.json`.

{% code title="PersonJson.json" %}

```javascript
{
  "Html": "/HelloWorld/PersonJson.html",
  "FirstName$": "",
  "LastName$": "",
  "SaveTrigger$": 0,
  "FullName": "",
  "Expenses": [{}],
  "NewExpenseTrigger$": 0,
  "CurrentBalance": 0,
  "CancelTrigger$": 0,
  "DeleteAllTrigger$": 0
}
```

{% endcode %}

## Add Buttons to the View

Now, let's add the buttons that will increment these values in the same way that our save and new expense button does now.

Delete button:

{% code title="PersonJson.html" %}

```markup
<button value="{{model.DeleteAllTrigger$::click}}" onmousedown="++this.value">Delete all expenses</button>
```

{% endcode %}

Cancel button:

{% code title="PersonJson.html" %}

```markup
<button value="{{model.CancelTrigger$::click}}" onmousedown="++this.value">Cancel</button>
```

{% endcode %}

We'll place the delete button at the bottom of the page and the cancel button next to the save button.

## Create Event Handlers

The next step is to build handlers to react accordingly. We will also do that similar to the way we did with the other buttons.

{% code title="PersonJson.json.cs" %}

```csharp
void Handle(Input.CancelTrigger action)
{
    Transaction.Rollback();
}

void Handle(Input.DeleteAllTrigger action)
{
    Db.SQL("DELETE FROM Expense WHERE Spender = ?", this.Data);
    this.Expenses.Clear();
}
```

{% endcode %}

`Transaction.Rollback()` simply rolls back the state of your application to where you last ran a `Transaction.Commit()`.

The `DeleteAllTrigger` handler deletes all the expenses for the current `Person` in the database and clears the `Expenses` property in the view-model

## Result

You can now run your application again and rollback any mistakes you make or delete all expenses.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe504c380-8680-11e7-9944-1f85270462c6%2Fresizedpart6.gif?generation=1503327411295174\&alt=media)

If you get any errors, you can check your code against the [source code](https://github.com/StarcounterApps/HelloWorld/commit/e8424d4129d79379154281174cd31ad236bd8e22).


# The Next Step

Great job finishing this tutorial!

Now you know plenty about how to use Starcounter. We hope that you're eager to learn more.

Here are some resources that might be useful in starting to create your own applications:

1. [KitchenSink](https://github.com/StarcounterSamples/KitchenSink) demonstrates how to create different UI patterns.
2. The [guides](/2.3.1/guides) gives an in-depth look into different Starcounter topics.&#x20;
3. [Starcounter sample apps](http://starcounter.io/sample-apps/) which you can mix, match, and edit however you like.

Good luck!


# Guides

Use the navigation on the left to learn how to use Starcounter's interfaces for databases, view-models and networking in your apps. We have made sure that you will feel at home by using APIs that are similar to those you already know.


# Database

## Introduction

In Starcounter, classes are tables and class instances are rows. The database objects live in the database from the beginning. This means that they are not serialized to the database, they are *created* in the database from the time you use the new operator. SQL queries will immediately see them. There is no concept of moving data to and from the database. This means that accessing a property on a database object (e.g myPerson.FirstName) reads the value from the database rather than from the normal .NET heap. This is possible as the data of the database lives in the RAM.\
Read more in [Creating database classes](/2.3.1/guides/database/creating-database-classes) and [Data manipulation](/2.3.1/guides/database/data-manipulation).

## Object Identity and Object References

Starcounter is a database that offers relational access, graph access, object oriented access and document access all rolled into one. We recommend using [object references](/2.3.1/guides/database/object-identity-and-object-references) (implicit keys) rather than primary keys and foreign keys (explicit keys) as object references.

## SQL

Each class marked with the `[Database]` attribute or inheriting a class marked with the `[Database]` attribute are available from [SQL queries](/2.3.1/guides/database/querying-using-sql). Object relational mapping  is not needed since classes and tables are the same in Starcounter.

## Relations

Using object references in your code instead of foreign keys, it is easier than ever to create [relations](/2.3.1/guides/database/relations) between objects.

## Inheritance

Starcounter allows any database object to [inherit from any other database object](/2.3.1/guides/database/inheritance).


# Database Classes

## Introduction

Marking a class in the code as a database class is done by setting the `[Database]` attribute. This class becomes a part of the database schema and all instances of the class are stored in the database.&#x20;

## Creating Database Classes

Database classes are, for the most part, created the same way as any other class. The main difference is under the hood; Public fields and public auto-created properties in the database class become database columns and properties with explicitly declared bodies, such as `FullName` in the example below become code properties, which are not stored as columns, but can be accessed in SQL queries:

```csharp
[Database]
public class Person
{
    public string FirstName { get; set; } // Column
    public string LastName { get; set; } // Column
    public string FullName => $"{FirstName} {LastName}"; // Property
}
```

Database classes have to be declared as `public`, otherwise Starcounter will throw `ScErrEntityClassNotPublic (SCERR4220)` in weave-time.&#x20;

### Properties and Fields

We recommend using auto-implemented properties instead of fields in database classes because Starcounter will only allow auto-implemented properties in future versions to reduce maintenance and make it easier to be cross-platform. For developers this also means that weave-time will be faster and that error messages for edge cases will be clearer.

#### Access Levels

Properties and fields have to be public, otherwise, `ScErrNonPublicFieldNotExposed` will be thrown with `ScErrCantBindAppWithPrivateData (SCERR2149)`.
&#x20;This also applies to properties with the [`Transient`](/2.3.1/guides/database/creating-database-classes#preventing-fields-from-becoming-database-columns) attribute.

#### Preventing Fields From Becoming Database Columns

Use the `Transient` attribute to exclude fields and properties from becoming database columns. Fields or properties with the `Transient` attribute will remain as regular .NET fields and properties and their values will be stored on the head and be garbage collected with the objects they belong to. These fields and properties can't be queried with SQL.

```csharp
[Database]
public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string FullName => $"{FirstName} {LastName}";
    [Transient]
    public int ProcessSessionID { get; set; }
    [Transient]
    public int ProcessSessionNumber { get; set; }
}
```

### Constructors

Constructors in database classes work the same way as they do in any other class. For example, this works as expected:

```csharp
[Database]
public class Person
{
    public Person(string name)
    {
        this.Name = name;
        this.Created = DateTime.Now;
    }

    public string Name { get; set; }
    public DateTime Created { get; set; }
}
```

### Column Limit

Database classes can have a maximum of 112 columns for performance reasons. Thus, this is not allowed:

```csharp
[Database]
public class LargeTable
{
    public string Column1 { get; set; }
    public string Column2 { get; set; }
    // ...
    public string Column113 { get; set; }
}
```

## Nested Classes

Nested database classes are not supported. The limitation is that inner database classes cannot be queried with SQL.

## Create Database Objects

Database objects are created with the native program code operator `new`. For example, consider the following database class:

```csharp
[Database]
public class Person
{
    public String FirstName { get; set; }
    public String LastName { get; set; }
}
```

To create a new instance of this class, the syntax `new Person()` would be used, like this:

```csharp
new Person()
{
    FirstName = "John",
    LastName = "Doe"
};
```

{% hint style="info" %}
All database write operations, such as creating new database objects have to be wrapped in a [transaction](/2.3.1/guides/transactions).
{% endhint %}

### Deserializing to Database Classes

When deserializing to a database class, the deserialization should be wrapped in a transaction since it creates a new database object:

```csharp
using Starcounter;
using Newtonsoft.Json;

namespace DeserializeDemo
{
    [Database]
    public class Person
    {   
        public string Name { get; set; }
    }

    class Program
    {
        static void Main()
        {
            DeserializePerson(@"{""Name"": ""Gimli""}");
        }

        public static void DeserializePerson(string json)
        {
            Db.Transact(() =>
                JsonConvert.DeserializeObject<Person>(json));
        }
    }
}
```

### Casting From Non-Database Class

It's not possible to cast from a non-database class to a database class. For example, this is not possible:

```csharp
public void UpdatePerson(ExternalApiModel data) 
{
    (data.ExternalApiPerson as Person).Name = "John";
}
```

Instead, database object creation should be done with the `new` operator.


# Data manipulation

## Introduction

There are three data manipulation statements in SQL92: `INSERT`, `UPDATE` and `DELETE`. `UPDATE` is not supported in Starcounter SQL, `DELETE` is available through `Db.SQL`, and `INSERT` is available with [reload](/2.3.1/guides/working-with-starcounter/unload-reload). Objects are otherwise created and updated in the programming code.

All modifications have to be wrapped in a transaction. These modifications are visible to other transaction after the changes have been commited.

## Create

Database objects are created with the native program code operator `new`:

```csharp
new Person()
{
    FirstName = "John",
    LastName = "Doe"
};
```

{% hint style="info" %}
Read more about creating database object on the [database classes page](/2.3.1/guides/database/creating-database-classes#create-database-objects)
{% endhint %}

## Update

A database object can be updated using the native program code assign operator `=`.

For example, instead of instantiating an object like in the example above, it's possible to create the object and then update its properties:

```csharp
var person = new Person();
person.FirstName = "John";
person.LastName = "Doe";
```

To update the `LastName` of all the `Person` objects in the database, they would be looped through and updated, like so:

```csharp
var people = Db.SQL<Person>("SELECT p FROM Person p");
foreach (var person in people)
{
    person.LastName = person.LastName.ToUpper();
}
```

## Delete

There are two ways to delete database objects:

1. Using the `Delete` method on an object
2. Using `DELETE FROM`

`Delete` is used for single objects and `DELETE FROM` is used for many objects.

They look like this:

```csharp
var john = new Person();
john.Delete();

Db.SQL("DELETE FROM Person");

Db.SQL("DELETE FROM Person WHERE Name = ?", "John");
```

`person.Delete()` will just delete `john` while `DELETE FROM Person` will delete all objects of the `Person` class.

To delete database objects that are bound to the view-model, the view-model object should be deleted before the database object is deleted.

### ScErrMemoryLimitReachedAbort (SCERR8036)

Deleting many records with `DELETE FROM` might breach the size limit for a single transaction which will cause Starcounter to throw `ScErrMemoryLimitReachedAbort`. This can be fixed by using `Delete` and splitting the deletion in smaller transactions:&#x20;

```csharp
var people = Db.SQL($"SELECT p FROM {typeof(Person)} p");
foreach (var person in people)
{
    Db.Transact(() => person.Delete());
}
```

Read more in the [kernel Q\&A](/2.3.1/guides/working-with-starcounter/kernel-q-and-a).


# Object Identity and Object References

## Introduction

In Starcounter, an object reference uses an implicit foreign key to an implicit primary key called `ObjectId` as well as allowing natural keys and public keys to exist and be joined on when needed.

## Object Identity

Each object has a `ObjectNo` and `ObjectID` property that can be accessed using SQL or the extension methods `Object.GetObjectNo()` and `Object.GetObjectID()`.

`ObjectNo` is a unique `ulong` that can be treated as a primary key. The value becomes available when a database object is created, and is never changed.

`ObjectID` is a [URL Base64](https://en.wikipedia.org/wiki/Base64#URL_applications) string representation of `ObjectNo`. It's friendly for sending over the Internet. For example, `ObjectID` for `ObjectNo=40023` is `"JxX"`. `ObjectID` may contain `[A-Za-z0-9-_]` characters. Since string comparison in SQL is case insensitive, SQL queries using `ObjectID` may return more than one result.

## Primary Keys and Foreign Keys

As each object already has a unique key in `ObjectNo`, we recommend adding other keys when you need them, such as public keys that are meant to be exposed to the end user. Objects should be referenced using an object reference rather than by a foreign key except if there is a clear need for natural key referencing.

## Retrieving from Object Identity

Objects are retrieved from identity with `Db.FromId`. It handles both `ObjectID` and `ObjectNo`.

A common use is in handlers like this where `Person` is a database object:

```csharp
Handle.GET("PersonList/{?}", (string objectId) =>
{
    return new PersonPage()
    {
        Data = Db.FromId<Person>(objectId)
    };
});
```

There are four overloads of `Db.FromId`:

```csharp
T Db.FromId<T>(string base64Id);
T Db.FromId<T>(ulong id);
IObjectView FromId(ulong id);
IObjectView FromId(string base64Id);
```


# Querying with SQL

## Introduction

Data stored in the database can be queried for with SQL queries. The query is passed as a string to the `Db.SQL` method. The allowed SQL syntax is covered in the [SQL](/2.3.1/guides/sql) section.

In addition to querying the database with SQL, you can also use LINQ by using the package [Starcounter.Linq](https://www.nuget.org/packages/Starcounter.Linq/). Documentation for Starcounter.Linq is in in the [Starcounter.Linq repository on GitHub](https://github.com/Starcounter/Starcounter.Linq),

## Return Types

SQL queries are executed with the `Db.SQL` method. If the SQL command is `SELECT`, the function returns `Starcounter.QueryResultRows<T> : IEnumerable<T>`, it otherwise returns `null`.

```csharp
Db.SQL("SELECT p FROM Person p"); // => QueryResultRows<Person>
Db.SQL("DELETE FROM Person"); // => null
```

`T` in `QueryResultRows<T>` is the type of the object retrieved if the whole object is retrieved, otherwise, it's `Starcounter.Query.Execution.Row`.

```csharp
Db.SQL("SELECT p FROM Person p"); // => QueryResultRows<Person>
Db.SQL("SELECT p.Name FROM Person p"); // => QueryResultRows<Query.Execution.Row>
```

We recommend avoiding `Starcounter.Query.Execution.Row` when possible and instead retrieve the whole object and filter out the needed properties with Linq.

```csharp
Db.SQL("SELECT p.Name FROM Person p"); // Not recommended
Db.SQL("SELECT p FROM Person p")
    .Select(p => new { p.Name }); // Recommended
```

In addition to traditional SQL, Starcounter allows you to select objects in addition to primitive types such as strings and numbers. Also, it allows you to use C# style path expressions such as `person.FullName`.

{% hint style="warning" %}
When writing queries with `Db.SQL`, keep in mind that there are [certain reserved words](/2.3.1/guides/sql/reserved-words) that should be escaped. That is done by surrounding the reserved word in quotation marks.
{% endhint %}

{% hint style="warning" %}
The `QueryResultRows` class is deprecated from Starcounter 2.3.1. Read about the changes here: <https://starcounter.io/reducing-magic-increasing-familiarity-obsoleting-queryresultrows/>
{% endhint %}

## Making Queries Less Fragile

Since the queries are strings, they are sensitive to refactorings, for example if a database class is renamed. To make the queries less fragile, `typeof` and `nameof` can be used:

```csharp
Db.SQL($"SELECT p FROM {typeof(Person)} p");
Db.SQL($"SELECT p.{nameof(Person.Name)} FROM {typeof(Person)} p");
```

If the query is executed many times, constructing the string every time can become a performance problem. In that case, the string can be constructed once and stored in a static property.

## Using Variables

SQL variables are represented by question marks (?) in the query string, and you pass the current values of the variables as parameters to the method `Db.SQL`.

```csharp
var employees = Db.SQL<Employee>(
  "SELECT e FROM Employee e WHERE e.FirstName = ?", "Joe");
  
foreach (var employee in employees)
{
  Console.WriteLine($"{employee.FirstName} {employee.LastName}");
}
```

`Db.SQL`takes an arbitrary number of variables as long as the number of variable values are the same as the number of variables in the query string. Otherwise, an `ArgumentException` will be thrown.

```csharp
var lastName = "Smith";
Employee manager; //Assume some value is assigned to the variable manager.

var employees = Db.SQL<Employee>(
  "SELECT e FROM Employee e WHERE e.LastName = ? AND e.Manager = ?", 
  lastName, manager);
  
foreach (var employee in employees)
{
  Console.WriteLine($"{employee.LastName}; {emp.Manager.LastName}");
}
```

Each variable has an implicit type depending on its context in the query string. For example, a variable that is compared with a property of type `string` will implicitly be of the type `string`. If a variable is given a value of some incompatible type, then an `InvalidCastException` will be thrown. All numerical types are compatible with each other.

{% hint style="info" %}
You can only use `?` for variables after the `WHERE` clause. You can't, for instance, use `?` to replace the class name of a query.
{% endhint %}

## Query Processing Error

If a query cannot be processed due to some syntax or type checking error then the method `Db.SQL` will throw the `SqlException` `ScErrSQLIncorrectSyntax (SCERR7021)`.

```csharp
try
{  
    var people = Db.SQL<Person>("SELECT e.NonExistingProperty FROM Person p");
    
    foreach(Person person in people)  
    {    
        Console.WriteLine(person.Name);  
    }
}
catch (SqlException exception)
{  
    Console.WriteLine("Incorrect query: " + exception.Message);
}
```


# Data Types

## Introduction

This page describes the expected data types of the Db.SQL query results for the database object properties, fields and arithmetic operations.

## Field and Property Data Types

Your object properties and fields may have the following data types (`DbTypeCode`):

* `Binary`
* `Boolean`
* `Byte`
* `DateTime`&#x20;
* `Decimal`
* `Double`
* `Int16`
* `Int32`
* `Int64`
* `object`
* `enum`
* `SByte`
* `Single`
* `String`
* `UInt16`
* `UInt32`
* `UInt64`

The data types `Boolean`, `Byte`, `DateTime`, `Double`, `Int16`, `Int32`, `Int64`, `SByte`, `Single`, `String`, `UInt16`, `UInt32`, `UInt64` correspond to the .NET data types with the same names.

### String

The `string`data type can store data up to 1 MB of encoded string data. Thus, all strings with a length of less than 270600 will fit into the string data type. Strings longer than 270600 might fit depending on string content.

### Decimal

The data type `Decimal` is stored as a 64-bit integer and has a precision of six decimals and a range between `4398046511104.999999` and `-4398046511103.999999`. Trying to set the `Decimal` data type to a more precise value or to a value outside of the range throws `ScErrCLRDecToX6DecRangeError (SCERR4246)`. In those cases, `Double` can be used if data loss is acceptable.

### Object

The data type `object` represents a reference to a database object, i.e. an instance of a class, directly or by inheritance having the `Database` attribute set.

### Binary

The data type `Binary` is for representing binary data up to 1 MB.

### Numerical Types

All signed integers, `Int64`, `Int32`, `Int16` and `SByte` are represented as `Int64` internally in Starcounter SQL. The unsigned integers, `UInt64`, `UInt32`, `UInt16` and `Byte` are represented as `UInt64`. The approximate numerical types `Single` and `Double` are represented as `Double`.&#x20;

### DateTime

`DateTime` is represented as an `Int64` of the number of .Net ticks from `DateTime.MinValue.Ticks`.

### Enum

`enum` is supported as a data type. It's stored as a number in the database. Queries on `enum` will return a number which can be cast to an `enum`.

```csharp
using System;
using System.Linq;
using Starcounter;

public enum House { Targaryen, Tyrell, Baratheon, Greyjoy };

[Database]
public class Person
{
    public House House { get; set; }
}

class Program
{
    static void Main()
    {
        Db.Transact(() =>
        {
            var person = new Person() { House = House.Tyrell };
            var house = Db.SQL("SELECT p.House FROM Person p").First();
            Console.Write(house); // => 1
            Console.Write((House)house); // => Tyrell
            Console.Write(Db.FromId<Person>(person.GetObjectID()).House); // => Tyrell
        });
    }
}
```

### Nullable Types

If you want to store `null` values for data types that essentially are value types, you can instead use the corresponding nullable data types:

* `Nullable<Boolean>`
* `Nullable<Byte>`
* `Nullable<DateTime>`
* `Nullable<Decimal>`
* `Nullable<Double>`
* `Nullable<Int16>`
* `Nullable<Int32>`
* `Nullable<Int64>`
* `Nullable<SByte>`
* `Nullable<Single>`
* `Nullable<UInt16>`
* `Nullable<UInt32>`
* `Nullable<UInt64>`

## Arithmetic Operations

The data type of the result of an [arithmetic operation](/2.3.1/guides/sql/comparison-predicates) is one of the following:

1. `Double` (representing approximate numeric values) \[highest precedence]
2. `Decimal` (representing exact numeric values)
3. `Int64` (representing signed integers)
4. `UInt64` (representing unsigned integers - the natural numbers) \[lowest precedence]

In general, the data type of the result of an arithmetic operation is the data type with the highest precedence of the data types of the operands.

However, in the following special cases you need a data type with higher precedence to appropriately represent the result:

* A subtraction between `UInt64`'s (unsigned integers) has a result of data type `Int64` (signed integer).
* A division between any combination of `UInt64`'s and `Int64`'s (unsigned and signed integers) has a result of data type `Decimal`

## Collections

It is possible to have collections in the database class if the collection has an explicitly declared body. For example, the following properties are allowed:

```csharp
public List<string> Branches => new List<string>(){ "develop", "master" };

public IEnumerable<Person> Friends => Db.SQL<Person>("SELECT p FROM Person p");
```

These properties and fields are not allowed:

```csharp
public string[] Names { get; set; }
public List<Person> People { get; }
public IEnumerable Animals;
```

The properties with explicitly declared bodies cannot be queried for with SQL, but they can be accessed from the application code after they have been retrieved from the database. If a `Person` class has the property `Friends` with a declared body, then `Friends` can be accessed like so:

```csharp
var person = Db.SQL<Person>("SELECT p FROM Person p").FirstOrDefault();
IEnumerable<Person> friends = person.Friends;
```


# Relations

## Introduction

Relations between database classes are modeled similar to how it's done in relational databases while maintaining a object oriented implementation with references both ways.

## One-to-Many Relations

In a relational database you implement a one-to-many relation between two tables using a primary key of the first table and a foreign key of the second table. In object oriented programming code, you typically implement a one-to-many relation with a collection of object references in the first class and an object reference in the second class.

Using Starcounter we recommend that you model a one-to-many relation as in a relational database, i.e. an object reference in the second class, but also add a property (or method) in the first class that returns a collection of object references. In this way you both support the relational model and the typical object oriented implementation at the same time.

In the code example below there is a one-to-many relation between the entities/classes `Department` and `Employee` regarding employment. This one-to-many relation is stored in the object references `Department` in the class `Employee`.

```csharp
[Database]
public class Department
{
  public IEnumerable Employees => 
    Db.SQL<Employee>(
      "SELECT e FROM Employee e WHERE e.Department = ?", this);
}

[Database]
public class Employee
{
  public Department Department { get; set; }
}
```

In the code above, all instances of the `Employee` class has a `Department` reference to the department where they are employed and all instances of the `Department` class has a `Employees` collection of all employees of that particular department. However, within a `Department` object the collection is not stored internally in any data structure but instead is represented by an SQL query.

## Many-to-Many Relations

In a relational database you implement a many-to-many relation between two tables as two one-to-many relations to an associative table. In object oriented programming code, you typically implement a many-to-many relation between two classes as one collection of object references in each of the two classes.

Using Starcounter we recommend that you model a many-to-many relation as in a relational database, i.e. introducing a new associative class, but also add properties (or methods) that returns collections of object references in the two original classes. In this way you both support the relational model and the typical object oriented implementation at the same time.

In the code example below there is a many-to-many relation between the classes `Person` and `Company` regarding shares of the company. To represent this many-to-many relation we introduce the associative class `Shares`.

```csharp
[Database]
public class Person
{
  public IEnumerable EquityPortfolio => 
    Db.SQL<Shares>(
      "SELECT s.Equity FROM Shares s WHERE s.Owner = ?", this);
}

[Database]
public class Company
{
  public IEnumerable ShareHolders => 
    Db.SQL<Shares>(
      "SELECT s.Owner FROM Shares s WHERE s.Equity = ?", this);
}

[Database]
public class Shares
{
  public Person Owner { get; set; }
  public Company Equity { get; set; }
  public Int64 Quantity { get; set; }
}
```

In the code above, all instances of the class `Shares` has one reference to the person who owns the shares and one reference to the company of which the shares are issued. All instances of the `Person` class have a collection of all companies (`EquityPortfolio`) of which the person has shares, and all instances of the `Company` class have a collection of all persons (`ShareHolders`) which are shareholders of the company.


# Inheritance

## Introduction

Any database object can inherit from any other database object. The `Database` attribute is inherited from base- to subclasses. Hence, any class that directly or indirectly inherits a class with the `Database` attribute becomes a database class.

## Example

In this example, both `PrivateCustomer` and `CorporateCustomer` become database classes due to them inheriting `Customer`:

```csharp
[Database]
public class Customer
{
   public string Name { get; set; }
}

public class PrivateCustomer : Customer
{
   public string Gender { get; set; }
}

public class CorporateCustomer : Customer
{
   public string VatNumber { get; set; }
}
```

The table `Customer` will contain all `PrivateCustomers` and all `CorporateCustomers`. So if there is a private customer called "Goldman, Carl" and a corporate customer called "Goldman Sachs", the result of `SELECT C FROM Customer c` will contain both of them.

## Base Classes

A base class contains all instances of all derived classes in addition to the instances with the its own exact type.

```sql
SELECT C FROM Customer C WHERE Name LIKE 'Goldman%'
```

Returns `[ { Name:"Goldman Sachs" }, { Name:"Goldman, Carl" } ]`

## Derived classes

```sql
SELECT C FROM PrivateCustomer C WHERE Name LIKE 'Goldman%'
```

Returns `[{ Name:"Goldman, Carl", Gender:"Male" }]`

```sql
SELECT C FROM CorporateCustomer C WHERE Name LIKE 'Goldman%'
```

Returns `[{ Name:"Goldman Sachs", VatNumber:"1234" } ]`

## Inheriting From Non-Database Classes

A database class cannot inherit from a class that's not a database class. This will throw an error when the application is weaved.

It's also not possible to cast a non-database class to a database class.


# Sharing data

## Introduction

Starcounter apps in the same code host share the same database. This means that if one app changes some data in the database, all other apps in the same code host will immediately see that change. This, together with Blending, makes it possible for two apps that have been developed separately to act as one.

To share data between the apps, they have to agree on a common database schema for the tables that they wish to share. This is done by defining a common data model in a separate assembly.&#x20;

## Creating a common data model

One of the templates provided with the Visual Studio extension can be used to create a starting point for a common data model. The template is called "Starcounter Class Library" and can be found under `Add New Project -> Visual C# -> Starcounter -> Starcounter Class Library`.

The database classes that will be shared across different apps can be added to this project.\
\
To conveniently share the common data model, it can be put in the same Visual Studio solution as the apps or uploaded to NuGet.

## Add to apps

To include this common data model in an application, add it as a reference. For it to properly work, the "Copy Local" property has to be set to false. This can be done in the "Properties" tab of the reference. This means that the common data model will be shared as a dynamic-link library (DLL) between the apps.\
\
When sharing a common data model across apps, the apps have to reference the same version of the DLL.

{% hint style="warning" %}
Don't extend the common data model or create database classes that reference the common data model in apps. Starcounter will then fail to read the data if one of the apps are not running.
{% endhint %}

{% hint style="warning" %}
When you run multiple apps with a common data model, restarting one app will not reload the common data model DLL. Thus, any changes in the common data model requires the database to be restarted.
{% endhint %}

## The Simplified data model

All the Starcounter [sample apps](https://github.com/Starcounterapps) use a preinstalled data model called "Simplified". This allows all these apps to share data. To utilize this data model and share data with the sample apps, load the data model into an app by referencing it and setting "Copy Local" to false.

## Packing apps with common data models

To create packages with `starpack`, the DLL must be in the package together together with the app executables.&#x20;


# Database Configuration

## Introduction

The database can be configured in three different ways: with the [staradmin CLI](/2.3.1/guides/working-with-starcounter/staradmin-cli), in the [Administrator](/2.3.1/guides/working-with-starcounter/administrator-web-ui), or directly in the configuration file.

## Configuration File

By default, the database configuration file is located inside the server directory `Databases\[DatabaseName]\[DatabaseName].db.config`. Further instructions on finding the database configuration file can be found on the [Configuration Structure](/2.3.1/guides/working-with-starcounter/configuration-structure) page.

The options in this file can be configured either directly in the file or when creating the database with the [Staradmin CLI](/2.3.1/guides/working-with-starcounter/staradmin-cli). Alternatively, a limited subset of the options can be configured from the [Administrator](/2.3.1/guides/working-with-starcounter/administrator-web-ui).

For configurations regarding the network gateway, check the [Network Gateway](/2.3.1/guides/network/network-gateway) page.

## Configuration Options

Here are the most important database configuration options with their default values:

1. Should edition libraries be loaded in databases:\
   `LoadEditionLibraries: true`
2. Should JSON responses be wrapped in application name:\
   `WrapJsonInNamespaces: true`
3. Should applications be forced to register handlers starting with application name prefix:\
   `EnforceURINamespaces: false`
4. Should JSON responses from multiple applications be merged:\
   `MergeJsonSiblings: true`
5. Should the request filters be enabled (previously middleware filters):\
   `RequestFiltersEnabled: true`
6. Should static files HTTP responses have a special header `X-File-Path` that contains a full path to the actual file on the server (default value is `False`). This HTTP header is often useful for debugging purposes, but should be disabled in production:\
   `XFilePathHeader: true`
7. The HTTP port bound to the database: `DefaultUserHttpPort: 8080`


# Comparing Database Objects

## Introduction <a href="#comparison-in-sql-queries" id="comparison-in-sql-queries"></a>

Database object can either be compared in SQL `WHERE` clauses or in programming code.&#x20;

## Comparison in SQL Queries <a href="#comparison-in-sql-queries" id="comparison-in-sql-queries"></a>

In SQL queries database objects can be compared either by the equals `=` operator or by the `ObjectNo` value. The direct comparison is always preferable.

```csharp
var products = Db.SQL<Product>(
    "SELECT p FROM Product p WHERE p.Customer = ?", customer);
```

```csharp
var products = Db.SQL<Product>(
    "SELECT p FROM Product p WHERE p.Customer.ObjectNo = ?",
    customer.GetObjectNo());
```

## Comparison Between Instances <a href="#comparison-between-instances" id="comparison-between-instances"></a>

Two instances of a database class can be compared either with the `Object.Equals` method or with the `ObjectNo` value. The `Object.Equals` method is the preferable way.

```csharp
var firstProduct = new Product();
var secondProduct = new Product();

var anotherFirstProduct = Db.FromId(firstProduct.GetObjectNo());

firstProduct.Equals(secondProduct); // false
firstProduct.GetObjectNo() == secondProduct.GetObjectNo(); // false
firstProduct.Equals(anotherFirstProduct); // true
firstProduct.GetObjectNo() == anotherFirstProduct.GetObjectNo(); // true
```

{% hint style="warning" %}
The equals `==` operator and the `Object.ReferenceEquals` method will always return `false` when comparing database objects.
{% endhint %}


# Referential Integrity and Constraints

## Introduction

Starcounter does not have complete support for referential integrity or constraints.&#x20;

Referential integrity can instead be achieved using commit hooks which allow the developer to ensure that the correct corresponding item is deleted or added when removing or committing an item to the database.

## Referential Integrity with Commit Hooks

Commit hooks should be implemented in a separate class and then registered when the application is started. Here's an example of that:

Let's say you have two DB classes: parent `Order` and child `OrderItem`:

```csharp
[Database]
public class Order
{
    public string Customer { get; set; }
    public DateTime Date { get; set; }
}

[Database]
public class OrderItem
{
    public Order Order { get; set; }
    public string Product { get; set; }
    public int Quantity { get; set; }
}
```

Here the commit hooks are declared in a separate class in a `Register` method:

```csharp
public class Hooks
{
    public void Register()
    {
        Hook<Order>.BeforeDelete += (sender, entity) =>
        {
            Db.SQL("DELETE FROM OrderItem oi WHERE oi.\"Order\" = ?", entity);
        };

        Hook<OrderItem>.CommitInsert += (sender, entity) =>
        {
            if (entity.Order == null)
            {
                throw new Exception(
                    "OrderItem.Order is not a null property");
            }
        };

        Hook<OrderItem>.CommitUpdate += (sender, entity) =>
        {
            if (entity.Order == null)
            {
                throw new Exception(
                    "OrderItem.Order is not a null property");
            }
        };
    }
}
```

In this code, we register the hooks by using `var hooks = new Hooks();` and `hooks.Register();`.

```csharp
public class Program 
{
    public void Main() 
    {
        Hooks hooks = new Hooks();

        hooks.Register();

        // This transaction will pass without errors.
        Db.Transact(() =>
        {
            var order = new Order()
            {
                Customer = "A customer",
                Date = DateTime.Now
            };

            var item = new OrderItem()
            {
                Order = Order,
                Product = "Starcounter license",
                Quantity = 10       
            };
        });

        // This transaction will fail with an exception
        // since the Order property of the OrderItem is NULL.
        Db.Transact(() =>
        {
            var item = new OrderItem()
            {
                Product = "Starcounter license",
                Quantity = 10
            };
        });

        // This transaction will delete all Order and OrderItem entries.
        // The OrderItem entries will be deleted by the BeforeDelete commit hook on the Order class.
        Db.Transact(() =>
        {
            Db.SQL("DELETE FROM \"Order\"");
        });
    }
}
```

Two things that are done here:

1. When making an `INSERT` or `UPDATE`, the commit hooks will check if the reference to the database object is set properly. If not, then it throws an exception and prevents the item from being updated or saved to the database.
2. Before deleting a parent entity it deletes the children of that entity.

### OnDelete

As an alternative to the `BeforeDelete` commit hook, you can use the Starcounter method `OnDelete`.

`OnDelete` works similar to the `OnData` and `HasChanged` callback methods that are explained on the [callback methods page](/2.3.1/guides/typed-json/callback-methods). It executes some code every time an instance of that class is deleted.

This is how it looks in code:

```csharp
[Database]
public class Foo
{
    public void OnDelete()
    {
        CalledWhenInstanceIsDeleted();
    }
}
```

If we create a new instance of the class `Foo` and then delete it by calling `fooInstance.Delete()` we would run `CalledWhenInstanceIsDeleted()`.

## Constraints

Database constraints define certain requirements that a database has to comply with.

Starcounter does not have database constraints as a part of its schema definition. As a consequence of this, you cannot define constraints in Starcounter the same way you would do with most SQL databases.

One of the most common constraints is the unique constraint which states that values in specified columns must be unique for every row in the table. This constraint can be set in Starcounter, even though it's not a part of the schema definition, by using [indexes](/2.3.1/guides/sql/indexes).


# SQL

## Introduction

Starcounter SQL follows the established standard SQL92 (ANSI INCITS 135-1992, R1998) to support easy data exchange with other databases and external tools.

## Calling SQL

There are two to use Starcounter SQL:

1. In programming code with `Db.SQL` as described in [Querying using SQL](/2.3.1/guides/database/querying-using-sql).
2. &#x20;In the interactive SQL browser of the [Starcounter Administrator](/2.3.1/guides/working-with-starcounter/administrator-web-ui).

The SQL in the Administrator and in code are not identical. For performance reasons, [literals](/2.3.1/guides/sql/literals) can't be used in the programming code with `Db.SQL` while it's available in the Administrator. Because of this, when using the examples in this documentation, remember to replace literals with [variables](/2.3.1/guides/database/querying-using-sql#using-variables) in programming code.

## Object Extensions to SQL

Starcounter SQL contains some extensions to the SQL92 standard to better deal with objects, since the standard SQL only supports relational databases. For these extensions we follow the Object Data Standard ODMG 3.0 (ISBN 1-55860-647-4). The object extensions in Starcounter SQL are:

* object references,
* [path expressions](/2.3.1/guides/sql/path-expressions).

In traditional SQL you can only refer to tables, columns, rows and fields of values. The concept of an "object" is represented by a row in a table. You refer to an "object" by values on its primary key which are some specified columns.

In Starcounter SQL you can refer to an object itself. For example, in query below the identifier `e` is an object reference.

```sql
SELECT e FROM Employee e
```

In a traditional SQL database, to get data from more than one type of object (table/class) you have to do a "join" of a number of tables. For example, query below gives you the names of the employees and the names of the departments where they work.

```sql
SELECT e.FirstName, d.Name FROM Employee e JOIN Department d ON e.DepartmentId = d.Id
```

In Starcounter SQL there is a more convenient way to get the same result by instead using a path expression as in query below. In that way, you can from one type of object (`Employee`) reach another type of object (`Department`) by object reference.

```sql
SELECT e.FirstName, e.Department.Name FROM Employee e
```

In object oriented programming the extent of a class is all object instances of that class. An extent of object instances corresponds to a table of rows in a relational database. Thus, `Employee` in the example queries above can either be regarded as the extent of the class `Employee` or as the table `Employee`.


# Identifiers

## Introduction

Identifiers are used to select classes and properties in queries.

An identifier of a class or property name in Starcounter SQL is a sequence of upper and lower case letters between A and Z, digits, and underscores where the first character can't be a digit.

Querying a database class or property that doesn't follow these restrictions will throw a `SqlException`:

```csharp
[Database]
public class Päron {}

Db.SQL("SELECT p FROM Päron p"); // SqlException
```

## Casing

Since Starcounter SQL is case insensitive, identifiers that are equal except for different casing are regarded to be the same identifier. Starcounter will throw `ScErrTypeNameDuplicate (SCERR4078)` if it finds two database classes where only the casing differs and `ScErrFieldsDifferInCaseOnly (SCERR4261)` if there are two fields or properties in a database where only the casing is different.

## Qualifying Classes in Different Namespaces

Starcounter automatically creates a database schema from the class definitions in the application program code. Thus, you could have several classes with the same name but in different namespaces. As a consequence, in Starcounter SQL you qualify a class name by specifying its namespace, not by specifying a database name or schema name. See example query below.

```sql
SELECT m.MyProperty FROM MyCompany.MyApplication.MyModule.MyClass m
```

As long as the class name is unique you do not have to specify the namespace. See example query below. However, in SQL statements in programming code we strongly recommend you to qualify all class names so the SQL statements will be guaranteed to still be valid when you add new classes.

```sql
SELECT m.MyProperty FROM MyClass m
```

## Reserved Words

If you have some identifier in your database schema that conflicts with some\
[reserved words](/2.3.1/guides/sql/reserved-words) in Starcounter SQL, you can tell the SQL parser that the term should not be interpreted as the reserved word by putting it inside double quotes, as in query below.

```sql
SELECT n."Left", n."Right" FROM Node n
```


# Path Expressions

## Introduction

In Starcounter SQL you can refer to any property of a database classes as long as there is a path from the database class specified in the `FROM` clause to that property.

A path expression is an arbitrary long sequence of identifiers separated by dots. The first identifier in the sequence should be an alias to an database class defined in the `FROM` clause of the `SELECT` statement. The following identifiers except the last one should have object references as return values. The last identifier may return any supported datatype. For example:

```sql
SELECT e.Manager.Department.Location.Street FROM Employee e
```

If an identifier in a path expression returns `null` then the complete path expression will return `null`.

Path expression can also be used in `WHERE` and `ORDER BY` clauses:

```sql
SELECT e FROM Employee e WHERE e.Manager.Department.Name = 'sales'
SELECT e FROM Employee e ORDER BY e.Manager.Department.Profit
```

## Wildcards

You can use a wildcard (\*) to select all properties of a database class:

```sql
SELECT * FROM Employee
SELECT e.* FROM Employee e
```

The above queries return all the properties of the `Employee` objects, while the below query returns references to the `Employee` objects themselves.

```sql
SELECT e FROM Employee e
```

## Cast Operation

In some path expressions you need to cast the type of a property.

For example, consider a data model like this:

```csharp
[Database]
public class Person
{
  public Person Father { get; set; }
}

[Database]
public class Employee : Person
{
  public Employee Manager { get; set; }
}
```

&#x20;If you want to select the manager of each person's father whenever such manager exists, then this is incorrect  since `Father` is of type `Person` and `Person` has no `Manager` property:

{% code title="Incorrect" %}

```sql
SELECT p.Father.Manager FROM Person p
```

{% endcode %}

However, if you cast `Father` to type `Employee` then you can continue the path expression with `Manager`:

{% code title="Correct" %}

```sql
SELECT CAST(p.Father AS Employee).Manager FROM Person p
```

{% endcode %}

If the object reference `Father` for some objects in the extent `Person` is not of type, or subtype of `Employee` , then this object reference can't be cast to `Employee` and the operation returns `null`.

The cast operation only supports casts between different types of database objects and not between different value types.


# Data operators

## Introduction

In Starcounter SQL, the most common operators on data are implemented.

## Arithmetic Operators

The standard arithmetic operators, plus (+ x), minus (- x), addition (x + y), subtraction (x - y), multiplication (x \* y) and division (x / y), are supported for all numerical types:

```sql
SELECT (e.Salary * 12) / 365 FROM Employee e
```

{% hint style="info" %}
For the expected datatypes of an arithmetic operation, see [Datatypes](/2.3.1/guides/database/datatypes).
{% endhint %}

## String Concatenation

String concatenation (x || y) is supported. See example below.

```sql
SELECT e.FirstName || ' ' || e.LastName FROM Employee e
```


# Joins

## Introduction

Starcounter supports inner joins, cross joins, left outer joins and right outer joins.

The default join type is inner join, so if you use the reserved word `JOIN`, the parser will interpret it as an inner join.

## Cross Join

A cross join is an inner join without join condition, and thus can be regarded as a special case rather than a separate type of join.

This query is interpreted as a cross join:

```sql
SELECT e, d FROM Employee e JOIN Department d
```

It can also be written as an explicit cross join:

```sql
SELECT e, d FROM Employee e CROSS JOIN Department d
```

## Inner Join

This query is interpreted as an inner join:

```sql
SELECT e, d FROM Employee e JOIN Department d ON e.Department = d
```

It can also be written as an explicit inner join:

```sql
SELECT e, d FROM Employee e INNER JOIN Department d ON e.Department = d
```

## Outer Joins

For left outer join and right outer join you may omit the reserved word `OUTER`.

This query returns all employees and their managers, including the employees that have no manager:

```sql
SELECT e1, e2 FROM Employee e1 LEFT JOIN Employee e2 ON e1.Manager = e2
```

This left outer join can also be written as a right outer join:

```sql
SELECT e1, e2 FROM Employee e2 RIGHT JOIN Employee e1 ON e1.Manager = e2
```


# Aggregates

## Introduction

Starcounter SQL supports five different aggregates: `AVG`, `SUM`, `COUNT`, `MAX`, and `MIN`. These can be used with grouping and conditions on groups with the `GROUP BY` and `HAVING` clauses.

## Example

```sql
SELECT AVG(e.Salary), MAX(e.Salary), MIN(e.Salary), e.Department
  FROM Example.Employee e
  GROUP BY e.Department
  HAVING SUM(e.Salary) > 20000
```

## Using Asterisk Shorthand With `COUNT`

The asterisk shorthand is treated as a literal in `COUNT`. Since `Db.SQL` doesn't support literals, using `COUNT(*)` in `Db.SQL` will throw  `ScErrUnsupportLiteral (SCERR7029)`.

&#x20;There are three ways to work around this:

```csharp
// This throws SCERR7029
Db.SQL("SELECT COUNT(*) FROM Person").First();
// Using an identifier instead of * works
Db.SQL("SELECT COUNT(p) FROM Person p").First();
// Db.SlowSQL supports literals, so it can be used
Db.SlowSQL("SELECT COUNT(*) FROM Person").First();
// Linq can also give you the number of rows
Db.SQL("SELECT p FROM Person p").Count();
```

&#x20;The first option of using an identifier to get the count best in most cases, both for versatility and performance.


# Comparisons and Logical Operators

## Introduction

Starcounter implements operators that can be used for filtering in `WHERE` clauses. They can, for example, check if a value is equal to another value or if a value matches a specific pattern.

## Relational Operators

### Equality

The comparison predicates "equal" (`x = y`) and "not equal" (`x <> y`) are supported for all data types. See for example query below.

```sql
SELECT e FROM Employee e WHERE e.FirstName = 'Bob'
```

{% hint style="warning" %}
Starcounter SQL is case insensitive, so "Bob" will also match with "bob", "BOB" and so on.
{% endhint %}

### Comparisons

The comparison predicates "less than" (`x < y`), "greater than" (`x > y`), "less than or equal" (`x <= y`) and "greater than or equal" (`x >= y`) are implemented for the data types `String`, `DateTime` and all numerical types. See example below.

```sql
SELECT e FROM Employee e WHERE e.LastName >= 'Smith'
```

Since a `DateTime` value represents a timestamp it is often necessary to compare it with a `DateTime` range. The query below returns all employees with a `HireDate` between `'2006-11-01 00:00:00.000'` and `'2006-11-01 23:59:59.999'`.

```sql
SELECT e.FirstName, e.HireDate FROM Employee e
  WHERE e.HireDate >= DATE '2006-11-01' 
  AND e.HireDate < DATE '2006-11-02'
```

## Logical Operators

There are three logical operators in Starcounter SQL, `AND`, `OR`, and `NOT`:

```sql
SELECT e FROM Employee e 
    WHERE e.FirstName = 'Bob' 
    AND e.LastName = 'Smith'
    
SELECT e FROM Employee e 
    WHERE e.FirstName = 'Bob' 
    OR e.FirstName = 'John'
    
SELECT e FROM Employee e 
    WHERE NOT e.FirstName = 'Bob'
```

{% hint style="info" %}
The logical operators `NOT` and `OR` usually imply that indexes can't be used in the execution of the query, and therefore these operators should be used restrictively.
{% endhint %}

## IS NULL

The comparison predicates "is null" (`x IS NULL`) and "is not null" (`x IS NOT NULL`) are implemented for all data types. See for example query below.

```sql
SELECT e FROM Employee e WHERE e.Manager IS NULL
```

## LIKE

The comparison predicate "like" (`x LIKE y [ESCAPE z]`) is implemented for the data type `String`. In the specified pattern (`y`) the underscore character (`'_'`) match any single character in the string, and the percent character (`'%'`) match any sequence (possibly empty) of characters in the string. See for example query below.

```sql
SELECT e FROM Employee e WHERE e.FirstName LIKE 'B_b%'
```

The optional third argument to the `LIKE` predicate is an "escape character", for use when a percent or underscore character is required in the pattern without its special meaning. This is exemplified in query below.

```sql
SELECT s FROM Share s WHERE s.Unit LIKE '\%' ESCAPE '\\'
```

## IS

The comparison predicate `IS` checks if an object can be cast to a given type. It has similar semantics to [`Type.IsAssignableFrom`](https://msdn.microsoft.com/en-us/library/system.type.isassignablefrom.aspx). Consider, for example, the following code:

```csharp
[Database]
public class Person
{
    public string FirstName { get; set; }
}

[Database]
public class Teenager : Person
{
}

[Database]
public class Child : Teenager
{
}

class Program
{
    static void Main()
    {
        Db.Transact(() =>
        {
            new Person { FirstName = "Bob" };
            new Teenager { FirstName = "Johnny" };
            new Child { FirstName = "Elsa" };
        });
    }
}
```

Here, the `IS` operator can be used to determine inheritance in the data model. For instance, to retrieve all instances that can be cast to `Teenager`, a query like this would be used:

```sql
SELECT p FROM Person p WHERE p IS Teenager
```

It returns the `Child` and `Teenager` instances "Johnny" and "Elsa" because they can both be cast as `Teenager`.


# Sorting

## Introduction

The result of a query can be ordered with the `ORDER BY` clause. The `ASC` and `DESC` keywords can be added after the clause to specify the order of the sort.

## Example

```sql
SELECT e.LastName, e.Salary 
    FROM Employee e 
    ORDER BY e.Salary DESC, e.LastName ASC
```

## Sorting and Indexes

When there is an index matching the sort specification in the `ORDER BY` clause then the result can be obtained without executing any sorting. We therefore for performance reasons strongly recommend that necessary indexes are declared when using `ORDER BY` clauses as shown in statement below.

```sql
CREATE INDEX EmployeeIndex ON Employee (Salary DESC, LastName ASC)
```

{% content-ref url="/pages/-KtAuTUdLcxsviWZfTBb" %}
[Indexes](/2.3.1/guides/sql/indexes)
{% endcontent-ref %}


# Fetch

## Introduction

The number of returned results can be limited with the `FETCH` clause:

```sql
SELECT e.LastName, e.FirstName
  FROM Employee e
  FETCH FIRST 5 ROWS ONLY
```

The only mandatory reserved word is the word `FETCH` as in the example below; the other reserved words are optional.

```sql
SELECT e.LastName, e.FirstName
  FROM Employee e
  FETCH 5
```

## Using Fetch with Other Clauses

The `FETCH` clause should be after the main part of the query possibly including an `ORDER BY` clause but before an `OPTION` clause including hints, see example below.

```sql
SELECT e.LastName, e.FirstName
  FROM Employee e
  ORDER BY e.FirstName
  FETCH 5
  OPTION INDEX (e MyIndexOnFirstName)
```

### Fetch with Offset

The `OFFSET` clause can be used to skip a number of rows before beginning to fetch the rows. This can be used to apply patterns like result pagination. `OFFSET 0` is the same as omitting the `OFFSET` clause.

```sql
SELECT e.LastName, e.FirstName
  FROM Employee e
  FETCH 5
  OFFSET 50
```

The standard `OFFSET` functionality typically used in RESTful web applications has a set of issues when the database is updated while data is being fetched. When using the standard `OFFSET` and data is updated, deleted or inserted between requests, the client will receive the same row twice (if a row already retrieved was inserted) or miss a row (if a row already retrieved was deleted).

Furthermore, `OFFSET` has performance limitations, since it is difficult to know which objects should be retrieved from each table to skip the requested number of rows in the result.

For the reasons outlined above, use [`OFFSETKEY`](/2.3.1/guides/sql/offset-key) instead of `OFFSET` when possible.


# Offset Key

## Introduction

Starcounter allows to retrieve query results in portions without need to keep an `Enumerator` (a cursor) with query result open. In addition to standard [OFFSET](/2.3.1/guides/sql/fetch) clause, Starcounter SQL is extended with `OFFSETKEY` clause, which re-creates the query enumerator and continues after the last retrieved record.

While a server side cursor can provide a snapshot isolation, the `OFFSETKEY` functionality is a good compromise. It provides for client side cursors with no server side state, just as `OFFSET` does, but without the duplicate or missing records common with `OFFSET`.

## How to Create Queries with Offset Key

You retrieve each portion of the query result by sending a new query, just as you do with a standard `OFFSET` query. Instead of a number indicating the position you should skip to, you instead retrieve a string value (the offset key) at the end of each portion. For the next portion, you provide the exact same query, but with the new key as a parameter value.

The query usually includes [FETCH](/2.3.1/guides/sql/fetch) clause, which limits each retrieval (each portion). Each query will will begin to retrieve result after the last fetched row from the previous portion, taking into account any deletions, changes or insertions.

The initial query must be identical with the subsequent queries apart from the `FETCH` and `OFFSETKEY` clauses and actual fetch and offset key values supplied. The first time the query is executed (i.e. for the first portion), the offset key value should always be set to `null` or omitted.

The `OFFSETKEY` clause is placed at the end of the `SELECT` statement and can be together with [FETCH](/2.3.1/guides/sql/fetch) clause, e.g.:

```sql
SELECT u FROM User u FETCH ? OFFSETKEY ?
```

The first time the query is used, you should supply the value `null` as the OFFSETKEY.

The input parameter is a string key, which is obtained on an enumerator to be re-created by calling method `GetOffsetKey()`. The string key for OFFSETKEY clause, *offset key*, can be also retrieved from the query already having OFFSETKEY clause.

## Query Limitation

You can't use `OFFSETKEY` with `ORDER BY` or `GROUP BY` clauses.

## Getting an Offset Key in Initial Query

The offset key is obtained by calling method `GetOffsetKey()` on enumerable or on enumerator, which is instance of `IRowEnumerator`. Getting offset key on enumerable can be done only if one enumerator was open.

The enumerator is obtained for initial query by calling standard interface method `GetEnumerator`, e.g.:

```csharp
IRowEnumerator<User> e = Db.SQL<User>("SELECT u FROM User u FETCH ? OFFSETKEY ?", 10, null).GetEnumerator();
```

An offset key is obtained by calling method `GetOffsetKey`, which has the following signature:

```csharp
byte[] GetOffsetKey();
```

The offset key is obtained at any valid state of enumerator, i.e., after `MoveNext` was called and was `true`:

```csharp
byte[] key = null;
using (IRowEnumerator<User> rows = Db.SQL<User>("SELECT u FROM User u").GetEnumerator())
{
    int i = 0;
    while (rows.MoveNext())
    {
        User u = rows.Current;
        ...
        i++;
        if (i == 3)
        {
          key = rows.GetOffsetKey();  
        }
    }
}
```

The offset key can be obtained after query with FETCH clause was enumerated. The offset key will be valid, if there are more rows exist after the fetched number of rows:

```csharp
byte[] key = null;
using (IRowEnumerator<User> rows = Db.SQL<User>("SELECT u FROM User u").GetEnumerator())
{
    while (rows.MoveNext())
    {
        User u = rows.Current;
        ...
    }
    key = rows.GetOffsetKey();
}
```

## Continue to Retrieve Data with an Offset Key Query

To be able to recreate the `Enumerator` and continue query execution, the query with offset key, *offset key query*, should be the same as the initial query, which was used to obtain the offset key, *original query*. Both query string and query variable values should be the same. Only `FETCH` and `OFFSETKEY` clauses can be different between queries. Note that `OFFSET` and `OFFSETKEY` clauses cannot be presented in the same query.

If an offset key query is not exactly the same as the original query (apart from the `FETCH` or `OFFSETKEY` clauses and  values), then Starcounter will throw an exception.

If the offset key query is the same as the original query and the offset key is not `null`, then the first row of the result of the offset key query will be the next row after the row of the original query, which was retrieved last before the offset key was gotten. The next row is defined for the moment when the offset key query is called. Thus if there were rows inserted after the last row of the original query, then the offset key query will retrieve them. Deleting the last row of the original query does not affect the result of the offset key query.

Query with `null` value for `OFFSETKEY` clause is equivalent to query with omitted `OFFSETKEY` clause.

If a new row, which has the same values as the last row of original query, (with or without deleting the last row) is inserted, then depending on its place in an index used in the query plan the offset key query will either start from it or after it and the last row.

## Example

```csharp
byte[] offsetKey  = null;
var accounts = Db.SQL<Account>(
   "SELECT a FROM Account a WHERE a.AccountId < ? FETCH ?", 100, 10);
    
using (var enumerator = accounts.GetEnumerator())
{
   while (enumerator.MoveNext()
   {
       Account account = enumerator.Current;
       Console.Write(account.AccountId + " ");
   }
   offsetKey = enumerator.GetOffsetKey();
}

if (offsetKey == null) 
   return;
   
Console.WriteLine();

var accounts = Db.SQL<Account>(
   "SELECT a FROM Account a WHERE a.AccountId < ? FETCH ? OFFSETKEY ?",
   100, 5, offsetKey);
   
using (var enumerator = accounts.GetEnumerator())
{
   while (enumerator.MoveNext()
   {
       Account account = enumerator.Current;
       Console.Write(account.AccountId + " ");
   }
   offsetKey = enumerator.GetOffsetKey();
}
...
```

If the database contains accounts with following AccountIds:

```
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ...
```

The code above will return:

```
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15
...
```

\
If the database contains accounts with following AccountIds:

```
10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 ...
```

The code above will return:

```
10 20 30 40 50 60 70 80 90
```


# Indexes

## Introduction

Indexes are essential to reach optimal performance. `CREATE INDEX` creates indexes and `DROP INDEX` removes indexes.

## Declaring Indexes

Indexes are created with `Db.SQL`, where query string contains an index declaration and has the form `CREATE [UNIQUE] INDEX indexName ON typeName (propertyName [ASC/DESC], ...)`.

It is recommended to declare indexes before any retrieval query is issued. **Note** that indexes should be declared **outside** a transaction scope.

In the examples (1), (2), (3), (4) and (5) below we declare indexes on different properties/columns of the class/table Employee.

```sql
(1) CREATE INDEX EmpFirstNameIndex ON Employee (FirstName ASC)

(2) CREATE INDEX EmpLastNameIndex ON Employee (LastName ASC)

(3) CREATE INDEX EmpSalaryIndex ON Employee (Salary)

(4) CREATE INDEX EmpManagerIndex ON Employee (Manager)

(5) CREATE INDEX EmpDepartmentIndex ON Employee (Department)
```

The default order, used when there is no order declared, is ASC (ascending). The order specified in the index declaration only matters when you want the result set to be sorted (ORDER BY).

You can declare indexes on properties/columns of the following datatypes (DbTypeCode): Boolean, Byte, DateTime, Decimal, Int16, Int32, Int64, Object, SByte, String, UInt16, UInt32, UInt64.

You can declare combined indexes on up to ten different properties/columns of a class/table. In the examples (6), (7), (8) and (9) we have some combined indexes on two properties/columns of the class/table Employee.

```sql
(6) CREATE INDEX EmpLastnameFirstNameIndex ON Employee (LastName ASC, FirstName ASC)

(7) CREATE INDEX EmpFirstNameLastNameIndex1 ON Employee (FirstName ASC, LastName ASC)

(8) CREATE INDEX EmpFirstNameLastNameIndex2 ON Employee (FirstName DESC, LastName ASC)

(9) CREATE INDEX EmpDepartmentSalaryIndex ON Employee (Department ASC, Salary DESC)
```

## Checking for Declared Indexes

The `indexName` must be unique. If you define the same name more than once you will get an exception. It is possible to check if an index was already created by issuing a query, which selects a record from table `Starcounter.Metadata.Index` with column `Name` equivalent to the index name as in the example below.

```csharp
if (Db.SQL("SELECT i FROM Starcounter.Metadata.\"Index\" i WHERE Name = ?", "EmpDepartmentSalaryIndex").FirstOrDefault() == null)
    Db.SQL("CREATE INDEX EmpDepartmentSalaryIndex ON Employee (Department ASC, Salary DESC)");
```

## Dropping Indexes

Existing indexes can be dropped from a database by query with syntax `DROP INDEX index_name ON table_name`. For example:

```csharp
Db.SQL("DROP INDEX EmpDepartmentSalaryIndex ON Employee");
```

## Recommendations

For all SELECT statements in your programming code, it is recommended to declare, when possible, indexes for:

* all conditions in WHERE clauses,
* all join conditions,
* all sort specifications in ORDER BY clauses.

An execution of the query (11) below can make use of an index on the property/column FirstName such as the index (1) above. It can also make use of combined indexes such as (7) or (8) where the first property/column of the index is FirstName. It can not make use of the combined index (6) where FirstName is not the first property/column of the index.

An execution of the query (12), where we have two equality comparisons on FirstName and LastName, can make use of any of the combined indexes (6), (7) and (8), where the two first properties/columns of the indexes are FirstName and LastName.

An execution of the query (13) can efficiently make use of the combined indexes (7) and (8) because the first property/column of the index is FirstName, on which we have an equality comparison, and the second property/column of the index is LastName, on which we have a range comparison. However, an execution of this query can not efficiently make use of the combined index (6) because the range condition on the first property/column LastName of the index makes it not possible to use the equality condition on the subsequent property/column FirstName.

An execution of the join in the query (14) can make use of the indexes (5) or (9), since we need an index to efficiently find all Employee objects/rows for a particular Department object/row.

An execution of the query (15) can make use of the combined index (6) to find all Employee objects/rows in the requested order without any sorting.

An execution of the query (16) can also make use of the combined index (6) to find all Employee objects/rows in the requested order without any sorting, since the index can be traversed in the reverse order.

An execution of the query (17) can make use of the combined index (7) in the reverse order to find all Employees objects/rows in the requested order without any sorting order. However, an execution of this query can not efficienlty make use of the combined index (8) since it neither in the normal nor the reverse order match the requested order.

```sql
(11) SELECT e FROM Employee e WHERE e.FirstName = ?

(12) SELECT e FROM Employee e WHERE e.FirstName = ? AND e.LastName = ?

(13) SELECT e FROM Employee e WHERE e.FirstName = ? AND e.LastName > ?

(14) SELECT e, d FROM Employee e JOIN Department d
         ON e.Department = d WHERE d.Name = ?

(15) SELECT e FROM Employee e ORDER BY e.LastName ASC, e.FirstName ASC

(16) SELECT e FROM Employee e ORDER BY e.LastName DESC, e.FirstName DESC

(17) SELECT e FROM Employee e ORDER BY e.FirstName DESC, e.LastName DESC
```

## Index Hints in Queries

You can give a hint to Starcounter on what index to use for a specific query. See [Hints](/2.3.1/guides/sql/query-plan-hints) for more information.

## Derived Indexes

The current version do not support derived indexes. You need to define index on the class you like to query. For instance, say we have the following structure:

```csharp
[Database]
public class LegalEntity
{
   public string Name { get; set; }
}

public class Company : LegalEntity
{}

public class Person : LegalEntity
{}
```

You will need to define index on Name for both Company and Person.

If index is defined on a database property for the base class, the query optimizer might choose to use it in queries on a child class of the base class, but this will require to filter out all instances, which are not of the child class. For example, index is created on `Name` only for `LegalEntity` and a query is submitted for `Company`, then if the query optimizer chooses to use the index, it will add a filter predicate, which checks that all objects from the index are instances of `Company`.


# Literals

## Introduction

Literals are used for queries in the [Administrator](/2.3.1/guides/working-with-starcounter/administrator-web-ui), they can also be used instead of [variables](/2.3.1/guides/database/querying-using-sql#using-variables) in programming code with `Db.SlowSQL` although, this comes with a heavy performance penalty.

## Boolean

A boolean literal can have one of the two values true and false, which are represented by the two reserved words `TRUE` and `FALSE`. See example below.

```sql
SELECT e FROM Employee e WHERE e.Commission = TRUE
```

## Numeric

There are three types of numerical literals `Int64`, `Decimal` and `Double`.

An `Int64` literal is described by its integer value, as in example below.

```sql
SELECT e FROM Employee e WHERE e.Salary = 5000
```

A `Decimal` literal is described by its numerical value including a decimal point, as in example below.

```sql
SELECT e FROM Employee e WHERE e.Salary = 5000.00
```

A `Double` literal is described by two numerical values, the mantissa and the exponent, separated by the character `E`. The mantissa may include a decimal point, but the exponent may not. See example below.

```sql
SELECT e FROM Employee e WHERE e.Salary = 5.0E3
```

## String

A string literal is a sequence of characters beginning and ending with single quote characters. To represent a single quote character within a String literal, you write two consecutive single quote characters, as in example below.

```sql
SELECT p FROM Photo p WHERE p.Description = 'Smith''s family'
```

## Date-time

A date-time literal is either described by the reserved word `DATE` followed by a `String` literal of the form `yyyy-mm-dd`, the reserved word `TIME` followed by a `String` literal of the form `hh:mm:ss[.nnn]` (the specification of milliseconds is optional), or the reserved word `TIMESTAMP` followed by a `String` literal of the form `yyyy-mm-dd hh:mm:ss[.nnn]`. See examples below.

```sql
SELECT e FROM Employee e WHERE e.HireDate = DATE '2006-11-01'
SELECT e FROM Employee e WHERE e.HireDate = TIMESTAMP '2006-11-01 00:00:00'
```

Note that all date-time literals in fact are timestamps, which means that the date-time literal in the first query above does not represent the date `'2006-11-01'` but in fact the first millisecond of that date `'2006-11-01 00:00:00.000'`. Consequently, above examples are equivalent.

## Binary

A binary literal is described by the reserved word `BINARY` and the binary value represented by a Hexadecimal string, as in example below.

```sql
SELECT d FROM Department d WHERE d.BinaryId = BINARY 'D91FA24E19FB065A'
```

## Object

Since Starcounter SQL supports object references, you also need a way to represent an object reference to a specific object, i.e. an object literal. Every object in a Starcounter database can be identified by its unique object-id-number. You describe an object literal by the reserved word `OBJECT` followed by the object's object-id-number, as in example below.

```sql
SELECT e FROM Employee e WHERE e = OBJECT 123
```


# Query Plan Hints

## Introduction

The Starcounter SQL optimizer decides the execution plan of an SQL-query. If you want to hint the optimizer that you prefer some particular join order or that you prefer some particular indexes to be used, you can do that in the OPTION clause at the end of the SQL statement.

## Specifying Join Order

To specify a preferred join order to use, you write JOIN ORDER (extent-alias-sequence) in the OPTION clause. You do not need to specify the order of all included extents in the extent-alias-sequence, only the ones for which you have a preferred join order. See example below.

```sql
SELECT d.Name, e.LastName FROM Department d 
  JOIN Employee e ON e.Department = d 
  WHERE e.FirstName = 'Bob' 
  OPTION JOIN ORDER (e,d)
```

If it is not possible to execute a query in the join order specified in the OPTION clause, which can be the case for outer joins, then the optimizer chose the join order to use. If you specify several join order hints only the first one will be considered.

## Specifying Index

To specify a preferred index to use for a particular extent/table, you write  INDEX (extent-alias index-name) in the OPTION clause. If some specified index does not exist then the optimizer choses another index if there is one. See example below.

```sql
SELECT e.FirstName, e.LastName FROM Employee e 
  WHERE e.FirstName = 'John' AND e.LastName = 'Smith' 
  OPTION INDEX (e MyIndexOnLastName)
```

You can specify an index to use for each extent in the SQL-query as in example  below. If you specify more than one index hint for a particular extent only the first one will be considered.

```sql
SELECT e, m FROM Employee e JOIN Employee m ON e.Manager = m 
  WHERE e.FirstName = 'John' AND e.LastName = 'Smith'
  AND m.FirstName = 'David' AND m.LastName = 'King' 
  OPTION INDEX (e MyIndexOnLastName), INDEX(m MyIndexOnFirstName)
```

## Specify Index and Join Order

You can specify both one index hint for each extent and one join order hint in the OPTION clause of a query, which is exemplified in example below.

```sql
SELECT e, m FROM Employee e JOIN Employee m ON e.Manager = m 
  WHERE e.FirstName = 'John' AND e.LastName = 'Smith' 
  AND m.FirstName = 'David' AND m.LastName = 'King' 
  OPTION JOIN ORDER (e, m), INDEX (e MyIndexOnLastName), 
  INDEX (m MyIndexOnFirstName)
```


# Reserved words

## Introduction

There are certain keywords in SQL that are marked as reserved. The reserved words have to be surrounded by double quotes when not meant as keywords.

## Reserved Words

These are the reserved words in Starcounter SQL in alphabetic order

`ALL`, `AND`, `AS`, `ASC`, `AVG`,\
`BY`, `BINARY`,\
`CAST`, `COUNT`, `CREATE`, `CROSS`,\
`DATE`, `DATETIME`, `DELETE`, `DESC`, `DISTINCT`,\
`ESCAPE`, `EXISTS`,\
`FALSE`, `FETCH`, `FIRST`, `FIXED`, `FORALL`, `FROM`, `FULL`,\
`GROUP`,\
`HAVING`,\
`IN`, `INDEX`, `INNER`, `INSERT`, `IS`,\
`JOIN`,\
`LEFT`, `LIKE`, `LIMIT`,\
`MAX`, `MIN`,\
`NOT`, `NULL`,\
`OBJ`, `OBJECT`, `OFFSET`, `OFFSETKEY`, `ON`, `ONLY`,\
`OPTION`, `OR`, `ORDER`, `OUT`, `OUTER`, `OUTPUT`,\
`PROC`, `PROCEDURE`,\
`RANDOM`, `RIGHT`, `ROWS`,\
`SELECT`, `STARTS`, `SUM`,\
`TIME`, `TIMESTAMP`, `TRUE`,\
`UNIQUE`, `UNKNOWN`, `UPDATE`,\
`VALUES`, `VAR`, `VARIABLE`,\
`WHEN`, `WHERE`, `WITH`.

The list of reserved words might be extended in later versions of Starcounter SQL. In particular some keywords in SQL92 might become reserved words in Starcounter SQL.

## Escaping Reserved Words

Reserved words cannot be used in queries directly. They have to be surrounded with double quotes as in example:

```sql
SELECT d FROM "DATE" d
SELECT o FROM "ORDER" o
```

Double quoting can be applied to any identifier, but only necessary for reserved keywords. It is important to double quote each identifier in identifier change, e.g.:

```sql
SELECT t FROM "Order"."Date" t
```

{% hint style="warning" %}
You can't use square brackets `[ ]` to escape reserved words in SQL
{% endhint %}


# Query for Database Classes

## Introduction

The class `ClrClass` can be used to retrieve the classes in the database.

## Getting All Classes

For example, if the goal is to get all the classes in the database, a query like this would be used in the Starcounter Administrator which is found at `http://localhost:8181/#/databases/default/sql` when Starcounter is running:

```sql
SELECT * FROM ClrClass
```

This would give back all the classes, including the built-in ones.

## Getting User-Created Classes

To get the user-created classes, use this query:

```sql
SELECT *
FROM ClrClass c
WHERE c.Updatable=true
AND c.UniqueIdentifier NOT LIKE 'Simplified%'
AND c.UniqueIdentifier NOT LIKE 'Concepts.Ring%'
AND c.UniqueIdentifier NOT LIKE 'Starcounter.%'
AND c.UniqueIdentifier NOT LIKE 'SocietyObjects%'
```

For example, if the query is run after following the steps in [part one](/2.3.1/tutorial/create-a-database-class) of the tutorial, it should look like this:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2F-L7x9Otvv8VPc6QUkfAW%2F-L7x9Pxg8V2xEJ-r0jmg%2Fquery-database-tables.png?alt=media\&token=6b4b1353-8d92-4444-a89d-cd1f0107813b)


# SQL Isolation Between Applications

## Introduction

Applications running in the same code-host are isolated on different levels:

* REST handlers and URIs namespace.
* SQL classes and objects.
* Static file resources.

The principle for SQL isolation is that the database classes of one application should not be visible to the database classes of another application running in the same code-host.

## Example

For example, the first application defines a database class `App1Class` in its namespace:

```csharp
namespace App1
{
    [Database]
    public class App1Class
    {
        public string App1ClassField { get; set; }
    }
}
```

In the second application, the class `App2Class` is defined:

```csharp
namespace App2
{
    [Database]
    public class App2Class
    {
        public string App2ClassField { get; set; }
    }
}
```

The first application is now able to access its own `App1Class` using full and short names:

```csharp
var result = Db.SQL("SELECT c FROM App1.App1Class c").FirstOrDefault();
result = Db.SQL("SELECT c FROM App1Class c").FirstOrDefault();
```

The same appplies to the second application with the class `App2Class`.

However, the first application will not be able to retrieve classes from the second application and vice versa. For example, the following code will throw an exception `Unknown class App2Class`:

```csharp
var result = Db.SQL("SELECT c FROM App2Class c").FirstOrDefault();
```

Classes defined in private application references, such as private libraries, are only accessible within the application that references them.

## Shared Library

If the first and second application are referencing the same library, for example "SharedDll", then both applications have access to classes and objects from this shared library, no matter which application created those objects:

```csharp
namespace SharedDll
{
    [Database]
    public class SharedDllClass
    {
        public string SharedDllClassField { get; set; }
    }
}
```

With this, both applications are able to query the `SharedDllClass`:

```csharp
var x = Db.SQL("SELECT c FROM SharedDllClass c").First;
var x2 = Db.SQL("SELECT c FROM SharedDll.SharedDllClass c").First;
```

The usa of shared libraries is a way for several applications to share the same class definitions. If you have several applications that are required to use the same classes, you will need to create a shared library and move all common class definitions there. In rare cases whenever this is not possible and you still need to have several applications accessing each other classes, you can reference other applications from you "main" application, so only one application is started.

## SQL Queries in the Administrator

Currently, the Starcounter Administrator only supports SQL queries with fully namespaced class names. In the above example, only the following queries are legitimate:

```csharp
"SELECT c FROM App1.App1Class c"
"SELECT c FROM App2.App2Class c"
"SELECT c FROM SharedDll.SharedDllClass c"
```


# Transactions

## Introduction

Starcounter uses transactions to ensure [ACID](http://en.wikipedia.org/wiki/ACID) compliance. This page describes how it works in theory, while [Short-Running Transactions](/2.3.1/guides/transactions/short-running-transactions) and [Long Running Transactions](/2.3.1/guides/transactions/long-running-transactions) explains how to use transactions.

## Achieving ACID Compliance

### Atomicity

As defined on [Wikipedia](https://en.wikipedia.org/wiki/Atomicity_\(database_systems\)), an atomic transaction "is an indivisible and irreducible series of database operations such that either all occur, or nothing occurs." Starcounter ensures atomicity by wrapping changes of one transaction within a transaction scope. The changes commit simultaneously at the end of the scope. If something interrupts the transaction before the end of the scope is reached, none of the changes will commit to the database.

### Consistency

A consistent DBMS ensures that all the data written to the database follow the defined contraints of the database. In Starcounter, this is solved by raising exceptions when an "illegal" action is carried out within a transaction, such as commiting non-unique values to a field that requires unique values. The exception will in turn make the transaction roll back so that none of the changes are applied.

### Isolation

To make transaction isolated, Starcounter uses [snapshot isolation](https://en.wikipedia.org/wiki/Snapshot_isolation). This means that when a transaction initializes, it takes a snapshot of the database and stores it in a transactional memory. This means that every transaction sees its own snapshot of the database. For example, a SQL query that executes before a transaction commits will not be able to see the changes made by the transaction because the changes are isolated to that transaction's snapshot of the database. This works no matter how large the database is.

### Durability

Durability ensures that commited transactions will survive permanently. Starcounter solves this by writing transactions to a transaction log after commits. Find more information about this at the [log retention](/2.3.1/guides/working-with-starcounter/log-retention) page.

### Concurrency Control

When many users write to the database at the same time, the database engine must ensure that the data keeps consistent using atomicity and isolation. For example, if an account reads 100 and you want to update it to 110 and another transaction is simultaneously reading a 100 and wants to update it to 120. Should the result be 110, 120 or 130? To resolve this, the transaction must be able to handle conflicts. The easiest way to do this is to use locking. If you want your database engine to serve large numbers of users and transactions, locking is slow and expensive and can lead to [deadlocks](http://en.wikipedia.org/wiki/Deadlock). Locking is efficient when you almost expect a conflict, i.e. when the probability is high that you will have a conflict. The slow nature of locking is that it always consumes time, even if there is no conflict. Another word for locking is 'pessimistic concurrency control'. A more efficient way of providing concurrency is to use 'optimistic concurrency control'. As the name implies, you don't expect a conflict, but you will still handle it. The concurrency control in Starcounter is **optimistic concurrency control**. It makes the assumption that conflicts between transactions are unlikely. The database can allow transactions to execute without locking the modified objects. If a conflict occurs, Starcounter will restart the transaction until it commits, or 100 times. For long-running transactions, the developer has to implement the retry functionality.


# Short-Running Transactions

## Introduction

Short-running transaction creates a transaction that commits to the database when the scope runs out. These transactions are ACID. There are two types of short-running transactions - synchronous and asynchronous.

## Db.Transact

`Db.Transact` is the simplest way to create a transaction in Starcounter. It declares a transactional scope and runs synchronously, as described on the [previous page](/2.3.1/guides/transactions). The argument passed to the `Db.Transact` method is a delegate containing the code to run within the transaction. In code, it looks like this:

```csharp
Db.Transact(() =>
{
    new Employee
    {
        FirstName = "Samwise",
        LastName = "Gamgee"
    };
});
```

\
Since `Db.Transact` is synchronous, it sometimes becomes a performance bottleneck. Starcounter handles this by automatically scaling the number of working threads to continue processing requests even if some handlers are blocked. The maximum number of working threads is the number of CPU cores multiplied by 254, so with four cores, there would be a maximum of 1016 working threads. When these threads are occupied, the next `Db.Transact` call in line will have to wait until a thread is freed. [Db.TransactAsync](https://docs.starcounter.io/2.3.1/guides/transactions/pages/-KtlHxoVbK5fODJ0awcW#db.transactasync) circumvents this.

`Db.Transact` is an implementation of `Db.TransactAsync` with a thin wrapper that synchronously waits for the returned `Task` object.

## Db.TransactAsync

`Db.TransactAsync` is the asynchronous counterpart of `Db.Transact`. It gives the developer more control to balance throughput and latency. The function returns a `Task` that is marked as completed when flushing the transaction log for the transaction. Thus, the database operation itself is synchronous while flushing to the transaction log is asynchronous.

&#x20;`Db.Transact` and `Db.TransactAsync` are syntactically identical:

```csharp
Db.TransactAsync(() => 
{
    // The code to run in the transaction
})
```

While waiting for the write to the transaction log to finish, it's possible to do other things, such as sending an email:

```csharp
Order order = null;
Task task = Db.TransactAsync(() =>
{
    order = new Order();
}); // Order has been added to the database

SendConfirmationEmail(order);

task.Wait(); // Wait for write to log to finish
```

This is more flexible and performant than `Db.Transact`, but it comes with certain risks; for example, if there's a power outage or other hardware failure after the email is sent but before writing to the log, the email will be incorrect - even if the user got a confirmation, the order will not be in the database since it was never written to the transaction log.&#x20;

If a handler creates multiple write transactions, use `Db.TransactAsync` and then wait for all transactions at once with `Task.WaitAll` or `TaskFactory.ContinueWhenAll`. Otherwise, the latency of the handler will degrade.

Consider the following example:

```csharp
for (int i = 1; i <= 100000; i++)
{
    Db.Transact(() =>
    {
        new Person();
    });
}

// Execute some code after commiting and flushing all transactions
```

The result of the following code is exactly the same as the code above, but it's executed many times faster:

```csharp
var tasks = new List<Task>();
for (int i = 1; i <= 100000; i++)
{
    tasks.Add(Db.TransactAsync(() =>
    {
        // All transactions up to this point have already been committed
        new Person();
    }));
}
Task.WaitAll(tasks.ToArray());
// Execute some code after commiting and flushing all the transactions
```

In this case, each iteration of the loop doesn't require the previous transaction to be flushed to the disk. The transaction execution is still synchronous.

### Limitations

With the `Db.TransactAsync` API, it's tempting to use async/await in applications. Syntactically it's possible, although it's not that useful due to these limitations:

1. &#x20;Using async/await is not possible in a handler body as Handler API doesn't support async handlers.
2. &#x20;No special measures have been taken to force after-await code to run on Starcounter threads, so manual `Scheduling.ScheduleTask` might be required (see [Running background jobs](/2.3.1/guides/transactions/running-background-jobs) for details).
3. &#x20;use async/await with caution as they may inadvertently increase the latency. Say if user code runs transactions sequentially, putting await in front of every `Db.TransactAsync` will accumulate all the individual latencies. The right stategy in this case is to make a list of tasks and then await them at once.

## Transaction Size

Code in `Db.Transact` and `Db.TransactAsync` should execute in as short time as possible because conflicts are likelier the longer the transaction is. Conflicts requires long transactions to run more times which can be expensive. The solution is to break the transaction into smaller transactions.


# Long running transactions

## Introduction

Starcounter supports long-running transactions. A long-running transaction, as its name indicates, spans over a longer period of time. To create a long-running transaction, use `Db.Scope(Action)` or `(new Transaction(...)).Scope(Action)` and attach it to a view-model.

When creating long-running transactions, it could contain objects that might be modified by other transactions during the transaction's lifetime. This is because the isolation level for long-running transactions is lower and it will therefore not detect conflicts. This is different from the behavior of `Db.Transact` and `Db.TransactAsync`, as described in the [Using Transactions page](/2.3.1/guides/transactions/using-transactions#dbtransact-and-dbtransactasync-usage).

A long-running transaction can be used on any scheduled thread, however, it cannot be used from more than one thread at a time. If two or more threads call `transaction.Scope(Action)`at the same time, an exception will be thrown.

&#x20;This document is divided into three parts:

1. &#x20;Creating and attaching long-running transactions
2. &#x20;Dealing with already attached long-running transactions in view-models
3. &#x20;Exceptions and how to solve them

## Create and Attach

### The Reason For Attaching View-Models to Transactions

A very important reason for associating view-models with transactions is that it allows you to put business logic in the database objects rather than in some code inside the form or some code that gets called when you save the form.

Let's say, for example, that the discount of an order changes when you change the quantity of any of its order items. The more you buy, the cheaper it gets. You would want this business logic should be visible to the order form presented to the user when he uses the mouse and keyboard to edit the order. To honor the DRY principle (don't repeat yourself) you should not have to repeat any business logic already in the domain model (i.e. database object). And if you put the code on the discount in the form, you would not handle the separation-of-concern very nicely as the rule is probably not connected with a certain piece of user interface or even a single specific application, but rather a generic business rule that should keep it's integrity no matter how many editors and different clients and application servers you throw at it.

By allowing the business objects to live inside a transactional scope, the form can view the world as if the changes are there while the outside world does not yet see them. When the form is saved, the transaction is committed or if the form is not saved, the transaction is simply aborted.

### Assigning JSON to a Transaction

Let's assume that you are composing an email in a mail program. You are entering a recipient that is not yet in your contact database. You would then create a new EmailAddress object and assign it to your email.

If the user elects to cancel the email, the EmailAddress should not be saved. If the user elects to send/save the email, the email address should be saved. But the EmailAddress is directly edited in the email form. How does Starcounter know that it should only be saved if the user saves the email?

A new transaction is created calling `Db.Scope` that takes a delegate to be executed as parameter. The transaction will then attach to the view-model when the (view-model) object is instantiated.

```csharp
Handle.GET("/email-client/new-email", () =>
{
  return Db.Scope(() => 
  {
    var emailPage = new EmailPage()
    {
      Data = new Email()
    }
    return emailPage;
  })
});
```

When Starcounter executes the `Handle` function or when it otherwise operates on the object set in the `Data` property, it will first set the current transaction scope to the transaction set in the `Transaction` property in the view-model or its nearest parent view-model.

Inside your form, the changes are all there and the information appears updated on the user screen. For the outside world, no unsaved changes are visible to disturb the consistency of your database.

### Using an Existing Transaction

Sometimes a transaction is already attached on another part of the view-model. To reuse it, it needs to be scoped before the new page is created.

```csharp
Handle.GET("/email-client/new-email", () =>
{
  var masterPage = Self.GET<MasterPage>("/email-client");
  masterPage.Transaction.Scope(() =>
  {
    masterPage.CurrentPage = new MailPage()
    {
      Data = new Email()
    };
  });
  return masterPage;
});
```

## Attaching a Transaction to an Existing JSON Object

If the part of the view-model that the transaction should attach to is already instantiated, for example a default value for a property of type `Json`, the transaction can attach manually.

Lets assume that in the previous example, the `CurrentPage` property was already instantiated.

```csharp
Handle.GET("/email-client/new-email", () =>
{
  var masterPage = Self.GET<MasterPage>("/email-client");
  masterPage.Transaction.Scope(() =>
  {
    masterPage.CurrentPage.AttachCurrentTransaction();
  });
  return masterPage;
});
```

### Sharing Transactions

A transaction can attach and be used on more than one instance in the view-model. When a transaction has a scope, all calls inside the scope will use the same transaction.

In this example the call to the second handler with uri `/email-client/email/{emailId}` will use the transaction created in the first handler.

```csharp
Handle.GET("/email-client/new-email", () =>
{
  var masterPage = Self.GET<MasterPage>("/email-client");
  Db.Scope(() =>
  {
    var email = new Email();
    var mailPage = Self.GET<MailPage>("/email-client/email/" + email.GetObjectID());
    masterPage.CurrentPage = page;
  });
  return masterPage;
});

Handle.GET("/email-client/email/{?}", (string emailId) =>
{
  Email email = Db.SQL<Email>("SELECT e FROM Email e WHERE ObjectId=?", emailId).First;
  return new MailPage()
  {
    Data = email
  };
});
```

Scopes are nested, so if in the example the second rest-handler, `Handle.Get("/email-client/email/{?}", ...)` would also declare a scope it will still use the same transaction as created by the caller, `GET("/email-client/new-email", ...)`.

### Making Sure to Create a New Transaction

To always create a new transaction, manually create and scope it:

```csharp
Handle.GET("/email-client/new-email", () =>
{
  var masterPage = Self.GET<MasterPage>("/email-client");
  var transaction = new Transaction(false, false);
  transaction.Scope(() =>
  {
    var email = new Email();
    masterPage.CurrentPage = Self.GET<MailPage>("/email-client/email/" + email.GetObjectID());
  });
  return masterPage;
});
```

### Handling Long-Running Transactions in View-Models

When inside a view-model that's attached to a long-running transaction, it's possible to commit and rollback changes.

The syntax for these are `Transaction.Commit()` and `Transaction.Rollback()`.

`Transaction.Commit()` commits changes to the database which means that they will become visible for other transactions.

`Transaction.Rollback()` rolls back the state of the view-model. For example, with a commit that's immidiately followed by a rollback, no changes will roll back. Consider this scenario instead:

```csharp
void Handle(Input.CreateEmailTrigger action) 
{
  Transaction.Commit();
  new Email()
  {
    Address = this.Address
  };
  Transaction.Rollback();
}
```

In this scenario, the new `Email` that's created would roll back and the state of the view-model would return to the previous commit.

Most sample apps uses `Commit` and `Rollback` to allow the user to save or cancel change like in the following example:

```csharp
partial class MailPage : Json, IBound<Mail>
{
  void Handle(Input.RecipientAddress action)
  {
    var emailAddress = Db.SQL<EmailAddress>("SELECT e FROM EmailAddress e WHERE Address = ?", action.value).First;
    if (emailAddress == null)
    {
      emailAddress = new EmailAddress() 
      { 
        Address = action.value 
      };
      Data.RecipientAddress = emailAddress;
    }
  }

  void Handle(Input.SaveTrigger action)
  {
    Transaction.Commit();
  }

  void Handle(Input.CancelTrigger action)
  {
    Transaction.Rollback();
  }
}
```

You can find an example of this in [step 6](https://app.gitbook.com/guides/tutorial/cancel-and-delete/README.md) of the tutorial.

## Exceptions

### ScErrIteratorClosed (SCERR4139)

Starcounter throws this exception when an iterator closes before finishing. An iterator will close if a long-running transaction commits, rolls back, or cancels while iterating.

This is the simplest way to close the iteration and throw the exception if we assume that this code is in a long-running transaction:

```csharp
foreach (var person in Db.SQL("SELECT p FROM Person p"))
{
    Transaction.Commit();
}
```

Due to this, we recommend not to execute code with side effects during the iteration since it might cause it to close. This is important when the developer does not have full control over the side effects, such as when making `Self.GET` calls that might have responses from other apps.

### ScErrTransactionModifiedButNotReferenced (SCERR4287)

`ScErr4287` is thrown when a long-running transaction that's not attached to a JSON object writes to the database without committing or rolling back at the end of the scope.

For example:

```csharp
using Starcounter;

[Database]
public class Person {}

class Program
{
    static void Main()
    {
        Db.Scope(() =>
        {
            new Person(); // ScErr4287
        });
    }
}
```

\
Here, a new object is written to the database but it's never committed or rolled back because long-running transactions don't automatically commit at the end of the scope. Starcounter throws an exception here to avoid confusion on what changes are commited. When using the long-running transaction, ensure that all writes are commited or rolled back before the end of the scope:

```csharp
using Starcounter;

[Database]
public class Person {}

class Program
{
    static void Main()
    {
        Db.Scope(() =>
        {
            new Person();
            // some more code

            Transaction.Current.Commit(); // Or Transaction.Current.Rollback();
        });
    }
}
```


# Using Transactions

## Introduction

Although short- and long-running transaction are similar in many ways, there are also some crucial differences. This page outlines how to choose what transaction to use and how to mix transactions.

## Choosing Transaction

It's important to chose the right transaction. In most situation, the choice is clear - if you are going to attach a transaction to a view-model, use a long-running transaction, otherwise, use a short-running transaction. When the choice is not clear, consider these factors:

### Side Effects

Since `Db.Transact` and `Db.TransactAsync` can run more than once because of conflicts, they should not have any side effects, such as HTTP calls or writes to a file. \`Db.Scope\` can have side effects, as long as it's not in an iterator.

### Rollbacks

The only way to rollback changes in `Db.Transact` and `Db.TransactAsync` is to throw an exception in the transaction. The alternative is to use `Db.Scope` with `Transaction.Rollback`.

### Conflics

If conflicts are likely, use `Db.Transact` or `Db.TransactAsync` because these handle conflicts while `Db.Scope` doesn't.

## Mixing Transactions

Transactions can be mixed as outer and inner transactions - one transaction wraps around the other. These are the possible combinations and their effects:

| Outer | Inner | Effect                                  |
| ----- | ----- | --------------------------------------- |
| Long  | Long  | Execute inner as part of outer          |
| Long  | Short | Execute inner as a separate transaction |
| Short | Long  | Not supported. Run-time error           |
| Short | Short | Execute inner as part of outer          |

Transactions can be mixed as outer and inner transactions - one transaction wraps around the other. These are the possible combinations and their effects:

### Long-Running in Long-Running

With a long-running transaction inside a long-running transaction, they act as if they were one transaction:

```csharp
[Database]
public class Person {}

[Database]
public class Animal {}

class Program
{
    static void Main()
    {
        Db.Scope(() =>
        {
            new Person();

            Db.Scope(() =>
            {
                Transaction.Current.Commit(); // Commits the Person
                new Animal();
            });

            Transaction.Current.Commit(); // Commits the Animal
        });
    }
}
```

### Short-Running in Long-Running

Short-running transactions in long-running transactions are executed separately:&#x20;

```csharp
using Starcounter;
using System.Linq;

[Database]
public class Person {}

[Database]
public class Animal
{
    public string Specie { get; set; }
}

class Program
{
    static void Main()
    {
        Db.Scope(() =>
        {
            new Person();

            Db.Transact(() =>
            {
                new Animal() { Specie = "Dog" };
            }); // Animal is commited to the database - transaction is done

            // The Animal committed can be accessed in the outer transaction
            var animal = Db.SQL("SELECT a FROM Animal a").First();

            // Rolls back the Person but not the Animal
            Transaction.Current.Rollback();
        });
    }
}
```

### Long-Running in Short-Running

Using long-running transactions in short-running transactions is not supported, it will throw `ScErrTransactionLockedOnThread (SCERR4031)`:

```csharp
Db.Transact(() =>
{
    Db.Scope(() => // SCERR4031
    {
        new Person(); 
    });
}); 
```

### Short-Running in Short-Running

Short-running in short-running transactions work the same as with long-running in long-running transactions: the inner transaction is executed as a part of the outer:

```csharp
using Starcounter;

[Database]
public class Person {}

[Database]
public class Animal {}

class Program
{
    static void Main()
    {
        Db.Transact(() =>
        {
            Db.Transact(() =>
            {
                new Person();
            }); // Person is not commited

            new Animal();
        }); // Animal and Person are commited
    }
}
```

## ScErrReadOnlyTransaction

&#x20;If an operation is done on the database without a transaction an exception will be thrown:

```
The transaction is readonly and cannot be changed to write-mode. (ScErrReadOnlyTransaction (SCERR4093))
```

&#x20;For example:

```csharp
[Database]
public class Person {}

class Program
{
    static void Main()
    {
        new Person(); // SCERR4093
    }
}
```

&#x20;To fix this, wrap the operation in a transaction:

```csharp
[Database]
public class Person {}

class Program
{
    static void Main()
    {
        Db.Transact(() => new Person());
    }
}
```


# Running Background Jobs

## Introduction

This article describes useful patterns to run background jobs in Starcounter applications which require access to the database. In regards to the current implementation in Starcounter 2.x, it is important to keep in mind how transactions and deleting of objects work and thus avoid jobs that run for a long time, or possible forever.

When you perform reads from the database, an implicit read-only transaction is created for you carefully by Starcounter. I.e., `Main` of Starcounter application, delegates passed to `Scheduling.RunTask` and URI handling delegates are all wrapped into such an implicit read-only transaction. The transaction allows you to do reads from the database. If a job inside that transaction is running forever, deleted objects will only be marked as deleted but not removed from the database until the code-host is restarted or until the delegate is done and some other transaction performs a commit afterwards. This can have implications of space (on disc) needed for the image-files since they will continue to grow even if objects are deleted.

What it means practically is that spawning background tasks by running an infinite loop with a timer inside `Scheduling.RunTask`, URI-handling delegates and `Main` isn't a good idea. This is not only preventing object from purging and will steal computing resources, but is also considered a bad pattern of a background job in general. The correct pattern uses inversion of control, when a timer object invokes the action, and is explained below. The main point is to keep transaction scopes short.

## Long-running Threads

There is often a case when long running threads are needed, for example, timer jobs, statistics gathering, external information retrieval, status information update, etc. For this matter the following pattern is recommended: all long-running tasks should be inside standard .NET threads, not Starcounter-related. However, inside these threads, whenever Starcounter operation should be performed (database access, operation on the session, etc.) a special Starcounter task should be scheduled. It depends if you want to run the database operation and wait for the result or just schedule a database operation that will be performed as soon as scheduler grabs the task: thus the synchronous parameter in the `Scheduling.RunTask()` (read below for more information). So the general rule is: Starcounter schedulers are limited resources and thus should run only short tasks. If something long-running can be done outside Starcounter schedulers - it should run as a standard .NET tasks/threads and not occupy the scheduler. Below is more information on Starcounter schedulers and tasks.

## Basic Information About Scheduling

Each Starcounter scheduler has a queue of tasks that are supposed to be run on this scheduler. Tasks are picked from the queue and executed. To put a task in a queue, the `Scheduling.RunTask` should be used. When scheduling a task, you can specify the scheduler number, and if the thread should wait for the task to be picked by scheduler and completed. Here is the signature of the `Scheduling.RunTask`:

```csharp
Task RunTask(
    Action action,
    Byte schedulerId = StarcounterEnvironment.InvalidSchedulerId)
```

where:

* `Action action`: procedure to execute on scheduler.
* `Byte schedulerId = StarcounterEnvironment.InvalidSchedulerId`: optional parameter to select the scheduler, on which the `action` is going to run.

To make the `Action` execute synchronously, use the `Wait` method:

```csharp
Scheduling.RunTask(() => { }).Wait()
```

To determine if current thread is on scheduler call `StarcounterEnvironment.IsOnScheduler()`. To determine the amount of schedulers in your database call `StarcounterEnvironment.SchedulerCount`. To get current scheduler id call `StarcounterEnvironment.CurrentSchedulerId` (in case if calling thread is not on Starcounter scheduler the value `StarcounterEnvironment.InvalidSchedulerId` is returned).

## Using a Timer

Running a short-lived job using some timer. In this example the .Net class `System.Timers.Timer` is used.

The following sample will execute a job every minute and do needed database operations and then exit, until the timer trigger again.

The same scheduler is used in these samples (scheduler 0) for simplicity but a better solution might be to schedule jobs on all available schedulers.

```csharp
using System;
using System.Timers;
using Starcounter;

namespace TimerSample
{
    class Program
    {
        private static Timer timer; // keep it to avoid the timer being GC:ed

        static void Main()
        {
            timer = new Timer(60 * 1000); // 1 minute interval
            timer.AutoReset = true;
            timer.Elapsed += OnTimer;
            timer.Start();
        }

        static void OnTimer(object sender, ElapsedEventArgs e)
        {
            // Schedule a job on scheduler 0 without waiting for its completion.
            Scheduling.RunTask(() =>
            {
                Db.Transact(() =>
                {
                        // Access database.
                }, false, 0);
            });
        }
    }
}
```

## Running a Separate Non-Database Thread

Running a separate (non-database) thread that regularly schedules jobs that access database instead of using a timer will work as solution for the first problem, deleting and purging objects, but have another issue with shutting down the codehost. This is due to lack of event that usercode can listen to when codehost is terminating.

```csharp
using System.Threading;
using Starcounter;

namespace StarcounterApplication4
{
    class Program
    {
        private static AutoResetEvent arEvent;

        static void Main()
        {
            arEvent = new AutoResetEvent(false);
            ThreadPool.QueueUserWorkItem(o => { RunForever(); });
        }

        static void RunForever()
        {
            while (true)
            {
                Scheduling.RunTask(() => // Schedule a job on scheduler 0
                {
                    Db.Transact(() =>
                    {
                            // Access database.
                    });
                    arEvent.Set(); // Signal job complete.
                });

                System.Threading.Thread.Sleep(1000);
                arEvent.WaitOne(); // Wait for the current job to finish
            }
        }
    }
}
```

**Note:** using `thread.Start()` instead of `ThreadPool.QueueUserWorkItem` will lead to the following entries in the Starcounter log and the shutdown will take longer time.

```csharp
Thread foreverThread = new Thread(new ThreadStart(RunForever));
foreverThread.Start();
```

> 20150928T084607 Warning sc://chrhol-pc/personal Starcounter.Server - User code process takes longer than expected to exit. (, PID=1271188, Database=default)

And then, finally:

> 20150928T084622 Error sc://chrhol-pc/personal Starcounter.Server - ScErrCodeHostProcessNotExited (SCERR10018): When asked to shut down, the user code process agreed to shut down, but the process didn't exit gracefully in time. Killing it.. (, PID=1271188, Database=default)\r\nVersion: 2.0.0.0.\r\nHelp page: <https://github.com/Starcounter/Starcounter/wiki/SCERR10018>.

## Exceptions in scheduled tasks

Exceptions in scheduled tasks are logged to the [Administrator log](/2.3.1/guides/working-with-starcounter/administrator-web-ui#log). For example, the following code will return "No exception":

```csharp
Handle.GET("/Hello", () =>
{
    try
    {
        Scheduling.RunTask(() => throw new Exception());
        return "No exception";
    }
    catch
    {
        return "Exception";
    }
});
```

This is logged to the console:

```
System.Exception: Exception of type 'System.Exception' was thrown.
   at StarcounterApplication1.Program.<>c.<Main>b__0_1() in C:\Users\User\source\
epos\StarcounterApplication1\StarcounterApplication1\Program.cs:line 15
   at Starcounter.DbSession.<>c__DisplayClass5_0.<RunAsync>b__0() in C:\TeamCity\BuildAgent\work\sc-11226\Level1\src\Starcounter\DbSession.cs:line 216
HResult=-2146233088
```

If you wait for the `Task`, the exception will be brought to the waiting thread and logged. For example, the following code will return "Exception" and the same exception as above will be logged:

```csharp
Handle.GET("/Hello", () =>
{
    try
    {
        Scheduling.RunTask(() => throw new Exception()).Wait();
        return "No exception";
    }
    catch
    {
        return "Exception";
    }
});
```

## Await with scheduled tasks

Since .NET pick the thread when using the `await` keyword, there's no way to ensure that the code after an awaited scheduled task is executed on a Starcounter thread or .NET thread.

```csharp
// Create a database object on the Starcounter thread
var person = Db.Transact(() => new Person());

// The await keyword lets .NET pick the thread
await Scheduling.RunTask(() => Thread.Sleep(2000));

// .NET might pick a .NET thread or Starcounter thread. If a 
// .NET thread is picked, this code will throw an exception
Db.Transact(() => person.Name = "John");
```

To prevent undeterministic exceptions like this, use the `Wait` method to wait for the task to finish execution.

```csharp
var person = Db.Transact(() => new Person());

// Wait for the task to finish with the Wait method
Scheduling.RunTask(() => Thread.Sleep(2000)).Wait();

// This is now guaranteed to be on a Starcounter thread
Db.Transact(() => person.Name = "John");
```

You can also use the task returned by `RunTask` to wait for the task when the Starcounter thread is needed.

```csharp
var person = Db.Transact(() => new Person());
Task task = Scheduling.RunTask(() => Thread.Sleep(2000));

// Call a method that doesn't require a Starcounter thread
SomeLongCalculation();

// Wait for the task to finish to get back on a Starcounter thread
task.Wait();

// Excute code that requires a Starcounter thread
Db.Transact(() => person.Name = "John");
```


# Commit Hooks

## Introduction

Commit hook is a logic flow control pattern similar to [trigger](https://en.wikipedia.org/wiki/Trigger) in relational databases. It enables to hook the [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) events per objects of particular class. For cases when an object is being created (with a `new` operator), updated (by writing to a field) and deleted (when `Delete`is called, and after the committed delete), additional event handlers of code might be added for execution.

## Example

```csharp
using System;
using Starcounter;

namespace TestHooks
{
   [Database]
   public class Hooked
   {
      public string state { get; set; }
   }

   [Database]
   public class YetAnotherClass
   {
      public int Stock { get; set; }
   }

   class Program 
   {
      static void Main()
      {
         Hook<Hooked>.BeforeDelete += (s, obj) =>
         {
            obj.state = "is about to be deleted";
            Console.WriteLine("Hooked: Object {0} is to be deleted", obj.GetObjectNo());
         };

         Hook<Hooked>.CommitInsert += (s, obj) =>
         {
            obj.state = "is created";
            Console.WriteLine("Hooked: Object {0} is created", obj.GetObjectNo());
            var nobj = new YetAnotherClass() { Stock = 42 };
         };

         Hook<Hooked>.CommitUpdate += (s, obj) =>
         {
            obj.state = "is updated";
            Console.WriteLine("Hooked: Object {0} is updated", obj.GetObjectNo());
         };

         Hook<Hooked>.CommitUpdate += (s, obj) => // a second callback
         {
            Console.WriteLine("Hooked: We promise you, object {0} is updated", obj.GetObjectNo());
         };

         Hook<Hooked>.CommitDelete += (s, onum) =>
         {
            Console.WriteLine("Hooked: Object {0} is deleted", onum);
            Hooked rp = (Hooked)DbHelper.FromID(onum); // returns null here
            // the following will cause an exception
            // Console.WriteLine("We cannot do like this: {0}", rp.state);
         };

         Hook<YetAnotherClass>.CommitInsert += (s, obj) =>
         {
            Console.WriteLine("Never triggered in this app, since it happens to get invoked inside another hook");
         };

         Hooked p = null;
         Db.Transact(() =>
         {
           p = new Hooked() { state = "created" };
         });

         Db.Transact(() =>
         {
            p.state = "property changed";
            Console.WriteLine("01: The changed object isn't yet commited", p.GetObjectNo());
         });

         Console.WriteLine("02: Change for property of {0} is committed", p.GetObjectNo());

         Db.Transact(() =>
         {
            Console.WriteLine("03: We have entered the transaction scope");
            Console.WriteLine("04: We are about to delete an object {0}, yet it still exists", p.GetObjectNo());
            p.state = "deleted";
            p.Delete();
            Console.WriteLine("05: The deleted object {0} is no longer be available", p.GetObjectNo());
            Console.WriteLine("06: Were are about to commit the deletion");
         });
         Console.WriteLine("07: Deletion is committed");
      }
   }
}
```

The output produced is as follows (accurate to [ObjectNo](/2.3.1/guides/database/object-identity-and-object-references)):

```
Hooked: Object 29 is created
01: The changed object isn't yet commited
Hooked: Object 29 is updated
Hooked: We promise you, object 29 is updated
02: Change for property of 29 is committed
03: We have entered the transaction scope
04: We are about to delete an object 29, yet it still exists
Hooked: Object 29 is to be deleted
05: The deleted object 29 is no longer be available
06: Were are about to commit the deletion
Hooked: Object 29 is deleted
07: Deletion is committed
```

Those familiar with .NET recognize Starcounter follows a convention of .NET [EventHandler](https://msdn.microsoft.com/en-us/library/db0etb8x.aspx) for commit hooks. Currently, the first argument of the callback isn't used. The second argument is a reference to an object being transacted (for create, update and pre-delete events) or an `ObjectNo` of the object which itself is already deleted (for post-delete event). As in the .NET convention one can have an arbitrary number of event handlers registered per event, which will be triggered in the order of registration on the event occurrence.

## Q\&A

**Why there are separate pre-delete (**`BeforeDelete`**) and post-delete (**`CommitDelete`**) hooks?**&#x20;

Remember that after object is physically deleted in the end of a successful transaction scope, you can no longer access it in a post-delete commit hook delegate. However you might still want to do something meaningful with it just around the moment of deletion. That is why the pre-delete hook is introduced. Note that a pre-delete hook triggers callback inside the transaction scope, but not in the end of transaction. It means that, in case a transaction has been [retried `N` times](/2.3.1/guides/transactions#concurrency-control), any pre-delete hook for any object deleted inside this transaction will also be executed `N` times, while all other hooks will be executed exactly once, right after a successful transaction commit. Thus, consider pre-delete hook behaving as a transaction side-effect.

**How much should commit hooks be used?**

In general, in situations where you can choose, we recommend to avoid using commit hooks. They introduce non-linear flows in the logic, hence producing more complicated and less maintainable code. Commit hooks is a powerful tool that should only be used in situations where benefits of using them overweight the drawbacks. One popular example is separate logging of changes in objects of selected classes.

**Can I do DB operations inside commit hooks?**&#x20;

The answer is "Yes", since all commit hooks relate to write operations (create/update/delete), thus there must always be a transaction spanning these operations, and all event handlers are run inside this transaction. For example, in `TestHooks` we create an instance of a class `YetAnotherClass` inside `CommitInsert`, but do not introduce a transaction scope around this line. The reason being for it is that there is already a transaction from `Main` which spans this call.

**Notes.**

1. It is currently not possible to detach commit hook event handlers.
2. CRUD operations introduced inside a hook are not triggering additional hooks. For instance, in `TestHooks` the insert hook for `YetAnotherClass` is never invoked, because the only place for it triggered is in `CommitInsert`, which is itself a commit hook.
3. It is recommended to avoid sync tasks in commit hooks. Instead, wrap the tasks in `Session.ScheduleTask` or `Scheduling.ScheduleTask`. In essence, when doing anything more than updating database objects, an asynchronous task should be scheduled for it. Otherwise, unexpected behavior might occur, such as `Self.GET` calls returning `null`.


# Typed JSON

## Introduction

Typed JSON are C# classes that are serializeable to JSON. This makes it easy to work with JSON documents in an object-oriented manner.

&#x20;The view-models are Typed JSON in Starcounter web apps.

[JSON-by-example](/2.3.1/guides/typed-json/json-by-example) defines it, the [Code-Behind](/2.3.1/guides/typed-json/code-behind) extends it, and it's [bound](/2.3.1/guides/typed-json/json-data-bindings) to database data.

## Content

This section describes how to use Typed JSON.

* [JSON-by-example](/2.3.1/guides/typed-json/json-by-example) and [Code-Behind](/2.3.1/guides/typed-json/code-behind) describes how to define the Typed JSON and extend it to allow for interactivity.
* [JSON Data Bindings](/2.3.1/guides/typed-json/json-data-bindings) explains how to initially add database data to the Typed JSON objects and then bind Typed JSON objects to database classes.
* [Callback Methods](/2.3.1/guides/typed-json/callback-methods) describes the use of callback methods for certain actions on the Typed JSON objects.
* [Responding With JSON](/2.3.1/guides/typed-json/responding-with-json) and [Accepting JSON in Requests](https://app.gitbook.com/guides/typed-json/accepting-JSON-in-requests/README.md) describes how to send and receive Typed JSON objects using HTTP.
* [Primitive Arrays and Single Value Types](/2.3.1/guides/typed-json/primitive-arrays-and-single-value-types) and [Typed JSON Internals](/2.3.1/guides/typed-json/typed-json-internals) covers topics that, in most cases, are not practically applicable but can still be useful in some niche cases.


# JSON-by-example

## Introduction

JSON-by-example defines Typed JSON objects.

It works by providing a sample instance of JSON that transpiles into Typed JSON classes. You can find these generated classes in the `obj > x64 > Debug` or `obj > x64 > Release` directory of the project with the filename extension `json.g.cs`.

JSON-by-example is useful for these reasons:

* It can double directly as JSON mockups
* It can express trees of objects and arrays
* It's easy to specify default values

## Create JSON-by-example

To create a Typed JSON class, choose `New item`, or use Ctrl + Shift + A in Visual Studio and then select `Starcounter -> Starcounter Typed JSON`. The created file contains an empty JSON object which is the JSON-by-example.

One of the simplest JSON-by-example files look like this:

{% code title="PersonPage.json" %}

```javascript
{
    "FirstName": "",
    "LastName": ""
}
```

{% endcode %}

Here, we set the value to an empty string to declare the type.

You create an instance of the generated Typed JSON with a normal constructor call: `new PersonPage()`.

Accesing the properties of Typed JSON is the same as with any C# object:

```csharp
var personPage = new PersonPage();
string name = personPage.FirstName; // Contains the value "", an empty string
```

## Default Values

It's simple to set default values in JSON-by-example. Building on the previous code example, it might look like this:

{% code title="PersonPage.json" %}

```javascript
{
    "FirstName": "Steven", 
    "LastName": "Smith"
}
```

{% endcode %}

By doing this, the JSON returned when creating a new `PersonPage` object will be `{"FirstName":"Steven","LastName":"Smith"}`:

```csharp
Handle.GET("/GetPerson", () =>
{
    return new PersonPage(); // {"FirstName":"Steven","LastName":"Smith"}
});
```

## Supported Datatypes

Typed JSON follows the specification of JSON, which means that objects, arrays and single values are all allowed. One difference is that when working with the C#-object and numbers we have the possibility to specify a more exact type. What in JSON is `Number`, splits up in `Int64`, `Double` and `Decimal`.

The following is a list of the tokens in JSON and the equivalence in C#:

| JSON              | C#      |
| ----------------- | ------- |
| `{ }`             | Object  |
| `[ ]`             | Array   |
| `"value"`         | String  |
| `123`             | Int64   |
| `true`/`false`    | Boolean |
| `1.234` and `2E3` | Decimal |

To specify the type of a member in JSON-by-example, define it in the code-behind:

```javascript
{
  "Value": 2E3 // will parse as decimal by default.
}
```

```csharp
partial class Foo : Json
{
    static Foo()
    {
        // Value should be of type double, not decimal.
        DefaultTemplate.Value.InstanceType = typeof(double);
    }
}
```

## Writable JSON Values

By default, all the values declared in JSON-by-example are read-only for the client. Any client-side change to a read-only property will result in an error.

To mark a specific value as writable by the client, add a dollar sign (`$`) at the end of the property name, e.g.:

```javascript
{
   "FirstName$": "",
   "LastName$": ""
}
```

## Trigger Properties

Trigger properties is a common use for writable JSON properties. They notify the code-behind that a change has happened. Here's an example of a trigger property:

```javascript
{
    "FirstName": "",
    "LastName": "",
    "SaveTrigger$": 0
}
```

An incrementation in `SaveTrigger$`

## The HTML Property

In all the [sample apps](https://github.com/StarcounterApps/), there is an "Html" property in every, or almost every, `.json` file. The value of this property contains the path to the corresponding HTML view which means that the middleware [HtmlFromJsonProvider](/2.3.1/guides/network/middleware#htmlfromjsonprovider) can locate this HTML view and send it to the client. This allows the developer to return a Typed JSON object from a handler and still return the corresponding view as well.


# Code-Behind

## Introduction

To create interactivity for the Typed JSON classes, code-behind classes can be added to extend existing Typed JSON. This is done using `.json.cs` files, which are partial definitions for Typed JSON classes.

## Create Code-Behind Files

To create a Typed JSON class with code-behind, choose `New item` in Visual Studio and then select `Starcounter Typed JSON with Code-behind`. By creating one of these with the filename "Person", two files will be created:

{% code title="PersonPage.json" %}

```javascript
{
}
```

{% endcode %}

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    partial class PersonPage : Json
    {
    }
}
```

{% endcode %}

## Handling Input Events

Consider this JSON object with a property that is [writable](/2.3.1/guides/typed-json/json-by-example#writable-json-values) from the client:

{% code title="PersonPage.json" %}

```javascript
{
  "FirstName$": ""
}
```

{% endcode %}

To observe changes to this property, the code-behind method `Handle` can be used:

{% code title="PersonPage.json.cs" %}

```csharp
partial class PersonPage : Json
{
    void Handle(Input.FirstName action)
    {
        if (action.Value == "Albert")
        {
            Message = "You are not allowed to enter Albert. There can be only one.";
            action.Cancel();
        }
    }
}
```

{% endcode %}

The `Handle` method gets called with a parameter of type `Input`. `Input` is the base class for events triggered by the client.

The `Input` class is auto generated per each JSON view-model. It provides the following properties and methods:

* property `Value` - contains the new value of the user input
* property `OldValue` - contains the current value of the user input
* property `ValueChanged` - boolean, true if the new value is different than the old value
* method `Cancel()` - reject the new value. It prevents the propagation of the new value to JSON as well as to the bound data object
* property `Cancelled` - boolean, true if the `Cancel()` method was called

To get many more examples of how interactivity is handled, take a look at the [KitchenSink repo](https://github.com/StarcounterApps/KitchenSink) where the most common UI patterns are demonstrated.

## Referring to Nested Objects

JSON-by-example might contain nested objects. For example:

{% code title="PersonPage.json" %}

```javascript
{
  "Name": {
     "FirstName$": "",
     "LastName$": ""
  },
  "FullName$": ""
}
```

{% endcode %}

Code-behind for the root level and `Name`-level can be provided as two separate partial classes. For example:

{% code title="PersonPage.json.cs" %}

```csharp
partial class PersonPage : Json
{
    void Handle(Input.FullName action)
    {
        var words = action.Value.Split(' ');
        this.Name.FirstName = words[0];
        this.Name.LastName = words[1];
    }

    [PersonPage_json.Name]
    partial class PersonPageName : Json
    {
        void Handle(Input.FirstName action)
        {
            var person = this.Parent as PersonPage;
            person.FullName = action.Value + " " + this.LastName;
        }

        void Handle(Input.LastName action)
        {
            var person = this.Parent as PersonPage;
            person.FullName = this.FirstName + " " + action.Value;
        }
    }
}
```

{% endcode %}

The attribute `[PersonPage_json.Name]` is used to hint what is the path in JSON-by-example that the partial class refers to.

As you might have noticed, accessing a child object from a parent object in code-behind is as simple as providing a path expression: `this.Name.FirstName = words[0]`. The child property (`this.Name`) is of known type (`PersonPageName`).

However, accessing a parent from a child requires casting (`var person = this.Parent as PersonPage`). This is because there might be various parents that employ this particular child. In general, using the `Parent` property is discouraged, because it breaks the single-direction data flow. Child should be controlled by the parent and not vice versa.


# Data Bindings

## Introduction

Properties declared in Typed JSON can be bound to either a property in the code-behind file or a CLR object that exposes one or more public properties. Properties that are bound will read and write the values directly to the underlying object with no need to manually transfer the values to the view-model.

## Default Bindings

### Binding to Database Objects

To bind a Typed JSON object to a database object, the `Data` property is used.

Consider the following JSON file:

{% code title="PersonPage.json" %}

```javascript
{
   "FirstName": "",
   "LastName": "",
}
```

{% endcode %}

To bind the Typed JSON object `PersonPage` defined above to a database class `Person`, the following code can be used:

{% code title="Program.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    [Database]
    public class Person
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public string FullName => FirstName + " " + LastName;
    }

    class Program
    {
        static void Main()
        {
            Db.Transact(() => // Add a new Person instance to the database
            {
                new Person()
                {
                    FirstName = "Steve",
                    LastName = "Smith"
                };
            });

            Handle.GET("/GetPerson", () =>
            {
                var person = Db.SQL<Person>("SELECT P FROM Person P").First; // Retrieve a database object from the database

                var json = new PersonPage() 
                {
                    Data = person // Bind the database object to the Typed JSON object
                };

                return json;
            });
        }
    }
}
```

{% endcode %}

The `PersonPage` object will now look like this: `{"FirstName":"Steve","LastName":"Smith","FullName":"Steve Smith"}`.

Starcounter recognizes that the properties in `PersonPage` and `Person` object have the same name and populates the values in the Typed JSON accordingly. This is the default way that bindings are created.

### Binding to Code-Behind Properties

In addition to binding to database objects, Typed JSON properties can also be bound to code-behind properties.

To accomplish what was demonstrated in the [previous example](/2.3.1#binding-to-database-objects) by using code-behind properties instead of database properties, the following code can be used:

{% code title="PersonPage.json" %}

```javascript
{
   "FirstName": "Steven",
   "LastName": "Smith",
   "FullName": ""
}
```

{% endcode %}

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    partial class PersonPage : Json
    {
        public string FullName => FirstName + " " + LastName;
    }
}
```

{% endcode %}

{% code title="Program.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    class Program
    {
        static void Main()
        {
            Handle.GET("/GetPerson", () =>
            {
                return new PersonPage(); // {"FirstName":"Steven","LastName":"Smith","FullName":"Steven Smith"}
            });
        }
    }
}
```

{% endcode %}

### Mixing Database and Code-Behind Bindings

When mixing database and code-behind bindings, the code-behind bindings take presidence. This means that if there is a database property and code-behind property with the same name, the code-behind property will be the one that's used.

## Different Types of Bindings

There are different settings that can be specified on each property or JSON object. Auto, Bound, Unbound or Parent.

### Auto

When the binding is specified as auto (which is the default setting) each property is matched against the code-behind and the data object (if any). If a code-property with the same name or with a name specified as binding is found the JSON property is treated as bound and getting and setting values to it will get and set the value on the underlying property.

If no matching property is found, the JSON property is treated as unbound and values will be stored in the JSON.

### Bound

When a JSON property is specified as bound a compiler error or runtime error, depending on if the type to bind to is specified, is thrown. A JSON property declared as Bound **must** match a property in the code-behind or data object.

### Unbound

An unbound JSON property will store the value in the JSON.

### Parent

The same setting as specified on a parent is used, which will be one of the above (auto, bound, unbound).

### Rules When Bindings are Created

1. If a code-behind file exists, a property is searched for there.
2. If a property was not found in the code-behind or no code-behind exists, a property in the data object is searched for.
3. If no property was found in steps 1 and 2 and the binding is set to `Auto`, the property will be unbound. If binding was set to `Bound` an exception will be raised.

## Modify Bindings

There are different ways to modify bindings so that they do not bind the default way.

All binding modifications are done in a static constructor in the code-behind file. Like so:

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    partial class PersonPage : Json
    {
        static PersonPage()
        {
            // Modifications go here
        }
    }
}
```

{% endcode %}

### Opting Out of Bindings

In some cases we want to make sure that a specific property is not bound. This can be achieved by setting the value of `Bind` to `null`;

For example:

{% code title="PersonPage.json.cs" %}

```csharp
partial class PersonPage : Json
{
    static PersonPage()
    {
        DefaultTemplate.FullName.Bind = null;
    }
}
```

{% endcode %}

Setting the `Bind` property to `null` in this case ensures that the Typed JSON property `FullName` is not bound to any database, or code-behind, property.

By applying this to the example in the [bindings to database objects section](/2.3.1#binding-to-database-objects), the resulting JSON would be `{"FirstName":"Steve","LastName":"Smith","FullName":""}`. Since the `FullName` property is not bound, it will not contain any value.

### Binding to Properties With Different Names

If a property should be bound to a property that has a different name than the property in the JSON, a binding value can be set.

For example, to bind the Typed JSON property `FirstName` to the database property `LastName` in the example in the [bindings to database objects section](/2.3.1#binding-to-database-objects) and vice versa to essentially switch the names around, the following code can be used:

{% code title="PersonPage.json.cs" %}

```csharp
public class PersonPage : Json
{
    static PersonPage()
    {
        DefaultTemplate.FirstName.Bind = "LastName";
        DefaultTemplate.LastName.Bind = "FirstName";
    }
}
```

{% endcode %}

The resulting JSON looks like this: `{"FirstName":"Smith","LastName":"Steve","FullName":"Steve Smith"}`.

### Binding to Custom Properties in Code-Behind

Since it is possible to [bind to properties with different names](/2.3.1#binding-to-properties-with-different-names), it is also possible to bind to custom properties in the code-behind. For example:

{% code title="PersonPage.json.cs" %}

```csharp
public class PersonPage : Json
{
    static PersonPage()
    {
        DefaultTemplate.FullName.Bind = "CustomFullName";
    }

    public string CustomFullName => FirstName + FirstName + " " + LastName; 
}
```

{% endcode %}

The resulting JSON looks like this with the example in the [bindings to database objects section](/2.3.1#binding-to-database-objects): `{"FirstName":"Steve","LastName":"Smith","FullName":"SteveSteve Smith"}`.

### Binding to a Deep Property

It is also possible to bind to deep properties by providing full path to the property. Here, the property `FriendName` is bound to the deep property `Friend.FirstName`.

{% code title="PersonPage.json" %}

```javascript
{
  "FirstName": "",
  "FriendName": ""
}
```

{% endcode %}

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    partial class PersonPage : Json
    {
        static PersonPage()
        {
            DefaultTemplate.FriendName.Bind = "Friend.FirstName";
        }
    }
}
```

{% endcode %}

{% code title="Program.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    [Database]
    public class Person
    {
        public string FirstName { get; set; }
        public Person Friend { get; set; }
    }

    class Program
    {
        static void Main()
        {
            Db.Transact(() =>
            {
                var friend = new Person() { FirstName = "Bilbo" };

                new Person()
                {
                    FirstName = "Steve",
                    Friend = friend
                };
            });

            Handle.GET("/GetPerson", () =>
            {
                Person person = Db.SQL<Person>("SELECT P FROM Person P WHERE P.FirstName = ?", "Steve").First;
                var json = new PersonPage();
                json.Data = person;
                return json;
            });
        }
    }
}
```

{% endcode %}

The resulting JSON from this example looks like this: `{"FirstName":"Steve","FriendName":"Bilbo"}`.

### Setting Type of Binding for Children

Typed JSON objects that contain children can specify how the type of bindings on its children.

By setting the property `BindChildren`, each child that don't specify it's own binding, i.e. use the `Parent` option will have the binding decided by the parent.

Setting the value in the code-behind from the `BindingStrategy` enum:

{% code title="PersonPage.json.cs" %}

```csharp
partial class PersonPage : Json
{
    static PersonPage()
    {
        DefaultTemplate.BindChildren = BindingStrategy.Bound;
    }
}
```

{% endcode %}

The enum has the following values: "Auto", "Bound", and "Unbound".

{% hint style="warning" %}
It is not allowed to set the value for this property to `BindingStrategy.UseParent`. An exception will be raised in this case.
{% endhint %}

## Specify Data Type

Denoting a specific type that is used as data-object is optional but comes with some benefits.

If the JSON object is static, that is all properties are known compile-time, you will get an error during compilation if bound properties are invalid, instead of getting a runtime error on first use. The Data property itself will also be typed to the correct type which means that you don't need to cast the data-object when using it.

### IBound

The JSON code-behind class has to implement `IBound<T>` to set custom data type.

{% code title="PersonPage.json.cs" %}

```csharp
[PersonJson_json]
public partial class PersonJson : Json, IBound<MyNamespace.Person>
```

{% endcode %}

**Note:** empty JSON objects do not have code-behind classes so it is not possible to declare a custom data type for them.

### IExplicitBound

`IExplicitBound` is an improved implementation of `IBound`. They are used the exact same way, though `IExplicitBound` allows more control over the bindings.

When using `IExplicitBound`, properties in JSON-by-example are expected to be bound. This allows the pinpointing of failed bindings which otherwise could go unnoticed. If the JSON-by-example looks like this:

{% code title="PersonPage.json" %}

```javascript
{
  "Name": "",
  "Age": 0,
  "Address": ""
}
```

{% endcode %}

And the database class looks like this:

{% code title="Person.cs" %}

```csharp
public class Person
{
  public string Name { get; set; }
  public long Age { get; set; }
  public string Address { get; set; }
}
```

{% endcode %}

If the code-behind includes `IExplicitBound` like this:

{% code title="PersonPage.json.cs" %}

```csharp
public class PersonPage : Json, IExplicitBound<Person>
```

{% endcode %}

Then it will compile successfully.\
If `public long Age` was removed, then the following error would be displayed: `'Person' does not contain a definition for 'Age'`. The reason for this is that `IExplicitBound` would look for a database property corresponding to `Age` and fail.

Since `IExplicitBound` expects all values to be bound to *something*, properties that are not intended to be bound have to be explicitly unbound. As noted above, it will not compile without this. A static constructor can be used in order to explicitly unbind these properties. This is how it would look:

{% code title="PersonPage.json.cs" %}

```csharp
static PersonPage()
{
    DefaultTemplate.Age.Bind = null;
}
```

{% endcode %}

Now, the code will compile successfully because it is explicitly described that the `Age` property will not be bound. This is further described in the section "Opt-out of Bindings".


# Callback Methods

## Introduction

Typed JSON files can be affected by changes from the client or the server. This page describes callback methods for changes on the server side.

## OnData

`OnData` is the method that is called when the `Data` property is set, as described in the [data bindings section](/2.3.1/guides/typed-json/json-data-bindings#binding-to-database-objects).

The `OnData` method is usually used to initialize the parts of the view-model that cannot be initialized by setting the `Data` property. Several examples of this can be found in the [KitchenSink repo](https://github.com/StarcounterApps/KitchenSink/blob/fad83975ec3b4ebf6201458ea228547e6756d507/src/KitchenSink/ChartPage.json.cs).

The basic structure for using `OnData` looks like this:

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    partial class PersonPage : Json
    {
        protected override void OnData()
        {
            base.OnData();

            // Code to be run when the Data property is set
        }
    }
}
```

{% endcode %}

## HasChanged

And the same goes for `HasChanged` - that method is always called when there is a change on a server side. But this time it indicates a change of a value in JSON root class.

> To capture a change in a child subtree, it is efficient to define another partial class and use HasChanged method there.

This method implemented in the same way as `OnData` - all the declaration is happening in the code-behind file.\
Unlike `OnData` the method `HasChanged` is not that commonly used but only when there is a need for auto-committed database transactions every time data updates.\
There is a quick example on `HasChanged` usage:

```csharp
using Starcounter;
using Starcounter.Templates;

namespace ModelChangeEventTestProject
{
    partial class Page2 : Json
    {
        protected override void HasChanged(TValue property)
        {
            base.HasChanged(property);
        }
    }

    [Page2_json.Property2]
    partial class Page2Property2 : Json
    {
        protected override void HasChanged(TValue property)
        {
            base.HasChanged(property);
        }
    }
}
```

Just to sum up methods purposes:

* `OnData` - triggered when the data property is changed.
* `HasChanged` - triggered when the value is changed.&#x20;


# Responding with JSON

## Introduction

Typed JSON objects are serialized automatically to the `application/json` format when returned from a handler.

## Example

{% code title="PersonPage.json" %}

```javascript
{
    "FirstName": "Bilbo",
    "LastName": "Baggins"
}
```

{% endcode %}

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    class Program
    {
        static void Main()
        {
            Handle.GET("/GetPerson", () =>
            {
                return new PersonPage(); // {"FirstName":"Bilbo","LastName":"Baggins"}
            });
        }
    }
}
```

{% endcode %}

## Setting Status Code and Description

The default HTTP status code for responses is 200 OK.

To change this, two methods are provided to the `Handle` class: `SetOutgoingStatusCode` and `SetOutgoingStatusDescription`.

In code, they look like this:

{% code title="Program.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    class Program
    {
        static void Main()
        {
            Handle.GET("/NotFound", () => 
            {
                Handle.SetOutgoingStatusCode(404);
                Handle.SetOutgoingStatusDescription("Not Found");
                return "";
            });
        }
    }
}
```

{% endcode %}

It is also possible to change the status code and description by creating a `Response` object:

{% code title="PersonPage.json" %}

```javascript
{
    "FirstName": "Gandalf",
    "LastName": "Gray",
    "Quote": "You shall not pass!" 
}
```

{% endcode %}

{% code title="PersonPage.json.cs" %}

```csharp
using Starcounter;

namespace MyApp
{
    class Program
    {
        static void Main()
        {
            Handle.GET("/GetPerson", () =>
            {
                var json = new PersonPage();

                var response = new Response()
                {
                    StatusCode = 403,
                    StatusDescription = "Forbidden",
                    Body = json.ToJson()
                };

                return response; // Body: {"FirstName":"Gandalf","LastName":"Gray","Quote":"You shall not pass!"}
            });
        }
    }
}
```

{% endcode %}

The JSON needs to be explicitly parsed to a string using `ToJson` when attaching a Typed JSON object to the body of a `Response`.


# Accepting JSON in Requests

## Introduction

Typed JSON objects can be used as a representation of the HTTP (or other REST protocol) request body payload. To accept a JSON object in a PUT message, simply declare a parameter with the type `Json`.

## Example

{% code title="PersonMsg.json" %}

```javascript
{
   "FirstName": "",
   "LastName": "",
   "Age": 0
}
```

{% endcode %}

{% code title="Program.cs" %}

```csharp
using Starcounter;

class Hello
{
   static void Main()
   {
      Handle.PUT("/hello/{?}", ( string name, PersonMsg message ) =>
      {
         return "Welcome " + name + " you are " + message.Age + " years old.";
      });         
   }
}
```

{% endcode %}

The parameter is not associated with the URI template, so the content of the body will be used to fill in the object.

Now you can call the above handler with a HTTP request, for example using `XMLHttpRequest` in a web browser or manually using cURL:

```
$ curl -X PUT -H "Content-Type: application/json"
-d "{FirstName:"Olle",LastName:"Svensson", Age:49}"
http://localhost:8080/hello/Olle

Welcome Olle you are 49 years old.
```


# Primitive Arrays and Single Value Types

## Introduction

You can create JSON-by-example that contains a primitive value, object, or array. In C#, all of these are handled the same way.

## Type Checking

&#x20;To check the type of a `Json`-instance, use one of these properties:

* `IsBoolean`
* `IsDecimal`
* `IsDouble`
* `IsInteger`
* `IsString`
* `IsObject`
* `IsArray`

```javascript
123
```

```csharp
var json = new SimpleIntegerJson();
Debug.WriteLine(json.IsInteger); // => true
Debug.WriteLine(json.IsString); // => false
```

## Getting and Setting Single Value Types

&#x20;To get or set values, use one of these properties:

* `BooleanValue`
* `DecimalValue`
* `DoubleValue`
* `IntegerValue`
* `StringValue`

```javascript
"simple string"
```

```csharp
var json = SimpleStringJson();
Console.WriteLine(json.StringValue); // => simple string
json.StringValue = "another string";
Console.WriteLine(json.StringValue); // => another string
```

&#x20;Trying to get or set to values of a different type will throw `InvalidOperationException`:

```javascript
"simple string"
```

```csharp
var json = SimpleStringJson();
Console.WriteLine(json.IsInteger); // => false
json.IntegerValue = 123; // InvalidOperationException
```

## Getting and Setting Primitive Arrays

&#x20;Values are added to arrays with the `Add` method. To get the values of an array, use `ToJson`.

```javascript
[ ]
```

```csharp
var json = new SingleArrayJson();

json.Add().IntegerValue = 1;
json.Add().StringValue = "foo";

Console.WriteLine(json.ToJson()); // [1, "foo"]
```

In the example above, the array holds values of different types. To restrict the array to one type, add a value of the type you want in the JSON-by-example. This value will not be included in the resulting JSON.

```javascript
[ 99 ]
```

```csharp
var json = new SingleArrayJson();

Console.WriteLine(json.ToJson()); // => []
json.Add().IntegerValue = 4;
json.Add().IntegerValue = 2;
Console.WriteLine(json.ToJson()); // => [4, 2]
json.Add().StringValue = "foo"; // InvalidOperationException
```

&#x20;Adding strings can be further simplified with an overload of `Add`:

```csharp
var json = EmptyArrayJson();
json.Add("foo");
Console.WriteLine(json.ToJson()); // => ["foo"]
```


# Typed JSON Internals

## Introduction

Typed JSON is a way to manipulate and work with JSON documents in an object oriented manner, with considerations for performance and ease of use. It is fully compatible with the standard JSON specification (<http://www.json.org/>) and can be converted to and from different representations, for example to and from JSON as a utf8 string.

## Templates and Instances

In its most basic form, Typed JSON consists of two parts, a template and an instance of the type `Json`. The template is an object that works as a schema that contains all properties and metadata for a Typed JSON object. This includes .Net types, names of the properties, databinding and more.

| Template class | .Net type  | Json type   | Comment                                             |
| -------------- | ---------- | ----------- | --------------------------------------------------- |
| TObject        | Json       | object      | A single JSON object of type `Json` or subclass     |
| TObjArr        | Arr\<Json> | array       | An array of JSON-objects. Arr is a subclass of Json |
| TArray\<T>     | Arr\<T>    | array       | An array of T, where T is `Json` or subclass        |
| TBool          | Bool       | true, false | Value                                               |
| TDecimal       | Decimal    | Number      | Value                                               |
| TDouble        | Double     | Number      | Value                                               |
| TLong          | Int64      | Number      | Value                                               |
| TString        | String     | string      | Value                                               |

*Table of the different templates that are available*

The templates also controls setting and getting values on the JSON instance. Each template have delegates that are created runtime, using Expression trees that are compiled to methods (<http://msdn.microsoft.com/en-us/library/bb397951.aspx>) to get and set values. Each delegate uses the correct .Net type directly so no casts or boxing of values will occur. This allow us to change the core behaviour on how values are stored and retrieved without changing the code around it and makes it easier to implement additional features.

The following delegates exists on each template. They are not always instantiated though, for example if databinding is not used the delegate for bound value will be null.

**Unbound value (UnboundGetter, UnboundSetter)**\
Used internally. Gets and sets values stored on JSON. For basic JSON objects the values will be stored in a .Net list directly on the instance.

**Bound value (BoundGetter, BoundSetter)**\
Used internally. Gets and sets values directly on the dataobject when databinding is used (databinding will be covered later)

**Bound or unbound value (Getter, Setter)**\
Can be used in usercode. Takes the most suitable bound or unbound delegate. Will always be available.

The delegates takes the JSON instance as parameter, and the setters an additional value parameter. In the example below a method on the instance is called, but what it does is simply to take the template and call the delegate with the instance as parameter.

`person.Set(tfullname, "John Doe");` will call `tfullname.Setter(person, "John Doe");`

### Example

In this example we create a template for a single JSON object, that describes a person, containing the properties FullName (string) and Age (Int64):

```csharp
using System;
using Starcounter;
using Starcounter.Templates;

namespace StarcounterApplication3
{
    class Program
    {
        static void Main()
        {
            TObject schema = new TObject();
            TString tfullname = schema.Add<TString>("FullName");
            TLong tage = schema.Add<TLong>("Age");

            Json person = new Json() { Template = schema };
            person.Set(tfullname, "John Doe");
            person.Set(tage, 33);

            Console.WriteLine(person.ToJson());
        }
    }
}
```

In the last line we convert the Typed JSON to a string-representation of the Typed JSON object. The output will be:\
`{"FullName":"John Doe","Age":33}`

## Comparing Bound Typed JSON Objects&#x20;

Whenever a data object is bound to a Typed JSON template, it checks whether the data object is the same as the previous data object. If they are, it checks any value differences and updates the JSON template accordingly. Otherwise, a new JSON instance will be created. Performance wise, only updating the value differences is faster.&#x20;

Reference or ID is used to determine if the existing and incoming data objects are the same. This works well when data objects are not coming from an external source. For example, if a data object is retrieved from a REST API and deserialized, it will always be considered a new object, even if the values are similar. Thus, a new JSON instance will always be created.

By overriding the default comparison, this can be solved. `SetDataObjectComparer` is an extension method on `TValue` that allows you to do that. It takes a `Func` with the parameters of the existing `Json` instance and the new data object and returns `true` if they're different and `false` otherwise. In this example, we override the default comparer by checking the value `SocialSecurityNum` to determine if it's a new object, in that way, a deserialized object would be classified the right way:

```csharp
using System;
using Starcounter;
using Starcounter.Templates;

[Database]
public class Person
{
    public string Name { get; set; }
    public long SocialSecurityNum { get; set; }
}

class Program
{
    static void Main()
    {
        Func<Json, object, bool> personComparer = 
            (Json instance, object obj) =>
        {
            var oldPerson = instance.Data as Person;
            var newPerson = obj as Person;

            if (oldPerson == null || newPerson == null)
                return false;

            return oldPerson.SocialSecurityNum == 
                   newPerson.SocialSecurityNum;
        };

        var schema = new TObject();
        schema.Add<TString>("Name");
        schema.Add<TLong>("SocialSecurityNum");

        schema.SetDataObjectComparer(personComparer);

        var personJson = new Json() { Template = schema };

        personJson.Data = Db.Transact(() =>
        {
            return new Person()
            {
                Name = "John",
                SocialSecurityNum = 188011201234
            };
        });
    }
}
```

## Populating Typed JSON from JSON

Typed JSON objects can be populated from JSON strings with the method `PopulateFromJson`. This is, for example, useful when receiving JSON from an external service.&#x20;

```javascript
  {
    "Name": ""
  }
```

```csharp
var person = new PersonPage();
person.PopulateFromJson("{ \"Name\": \"John\"}");
Console.WriteLine(person.ToJson()); // => {"Name":"John"}
```

The JSON has to be valid and the properties have to match the properties in the Typed JSON schema. Otherwise, Starcounter throws either `ScErrInvalidJsonForInput (SCERR14007)` or `ScErrJsonPropertyNotFound (SCERR14003)`.\
\
There's also an overload of `PopulateFromJson` that takes a byte array and a source size instead of a string.


# Blendable Web Apps

## Introduction

Starcounter apps are small, individually functional web apps that can easily interoperate with other Starcounter apps. There are a couple of traits that belongs to these apps:

1. They [do one thing and do it well](https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well)
2. They can run simultaneously with other apps.
3. They can share data with other apps.
4. They can share screen with other apps.

## Blending

Blending is the concept of making several apps look as one without apps knowing about each other. This means that apps can be developed separately and still be used together without touching the source code. In practice, a developer can build an app that does one thing and does it well and then run that app with multiple other apps in different combinations and still make all the apps look as one. It allows reuse on a new level.

Blending is achieved with two different technologies: [Attaching](/2.3.1/guides/web-apps/view-attaching) and [Composing](/2.3.1/guides/web-apps/view-composing).

Attaching handles requests from the browser to allow multiple apps to respond to the same request without the apps being aware of it. This means that the browser will receive one response containing the combined responses of multiple apps.

When a response containing multiple responses is received by the browser, the results are stacked on top of each other - it still looks like they are from different apps, even if they come from the same response. Composition takes the response and composes the layout to fit to that particular set of apps. Thus, the layout can be adapted to look like there's only one app on the screen.

{% hint style="info" %}
The code examples in this section are built on modern web technologies that are supported in the current versions of Chrome, Firefox, Safari, Edge, and Opera. Other web browsers are currently not supported.
{% endhint %}


# Starcounter MVVM

## Introduction

If you have used MVC or MVVM frameworks in database applications, it can be worth noting that Starcounter does things a little differently.

## Physical Tiers

Most Starcounter applications have two physical tiers - the client and the server.

## Logical Tiers

There are typically three logical tiers in a Starcounter application. The domain model, view-model and view. All three tiers are different from what you might be used to.

### Tier 1 - The Domain Model

In most database systems, the back end tier is a database management system (DBMS) and the next tier is an application layer that reads and writes data to the DBMS. In Starcounter, these two tiers are merged.

The database image master uses RAM instead of disk. The computer resources needed to move data to and from the database image are greater than actually executing the business logic. Because all data is already in RAM, the Starcounter application layer and the database layer is one and the same. In this way, programming becomes simplified and performance gets better.

### Tier 2 - The View-Model

Unlike most database systems, the view-model is mirrored between the client and the server. The server is the owner of the view-model and is the single source of truth. This means that the application logic on the server can change the view-model locally and the view is updated accordingly.

While it might sound expensive to keep a copy of each clients view on the server, all data on the server, including all the database data is already in RAM so very little new RAM needs to be allocated to keep track of what is going on. After all, more data exists in the database than is currently on screen on the clients at any given moment for most business applications.

Starcounter view-models are [Typed JSON classes](/2.3.1/guides/typed-json) defined by the server. Although JSON is merely a text based notation format for object trees, the Typed JSON implementation is a dynamic runtime tree that allows you to add, remove and change objects.

#### What is Typed JSON?

Typed JSON is a multipurpose vehicle. It supports the six data types of JSON (object, array, boolean, number, string and null). It typically has two uses. The first is to create and receive JSON messages as you would typically do in a REST style service or client. The other is to represent live view-models that can be bound to a user interface (client) or a controller (server).

#### Schema-less or Schema-full

Typed JSON allows dynamic (expando-like) creation of JSON trees. In addition, Typed JSON allows you to specify schemas. This can be done in two ways. One way to create a schema is to use the Typed JSON API. Another more innovative way is to provide an example JSON file with a sample instance of a tree. This method is referred to as JSON-by-example and provides you with an intuitive way to view you schema and share it between nodes (clients and servers).

#### Data Binding

In addition to allowing the developer to create object trees and assign properties one-by-one, Typed JSON also supports binding data using a special `Data` property. In this way, you can assign database results to populate JSON. The data bindings are two-way bindings such that changes to the data object is reflected in the Typed JSON and the changes to the Typed JSON is propagated to the data object.

Server-side view-models leverage the JSON capabilities of Starcounter. You can create a Typed JSON on the server that you send to the client **and** keep on the server. This means that the server can have direct access to the view-model from its controller logic. This is a more performant and secure way to deal with application logic as the client is left only to deal with the view (presentation logic).

### Tier 3 - The View

Because view-models are shared between the client and server, view rendering is performed on the client. In this way, you get better performance and as the number of users grow, so does the number of CPUs that renders the user interfaces.

When the view-model changes, the UI changes and when the user interacts with the UI, the model changes. In the simplest cases, this is often referred to as data based templating - you have some JSON data and you inject HTML into the DOM based on that data.


# Palindrom

## Introduction

Palindrom is a library that implements a radically simple, standards-compliant protocol for thin client web apps. It allows to create Single Page Applications with zero logic on the client side.

## Palindrom Protocol

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5f739d0-8680-11e7-9944-1f85270462c6%2Fpalindrom-flow.png?generation=1503327413311803\&alt=media)

Starcounter is embracing web standards to allow stateful, thin client web apps done by existing web standards. The application state is simply kept in JSON. Any changes to it, coming from the client or the server, are expressed in automatically generated JSON-Patch ([RFC 6902](http://tools.ietf.org/html/rfc6902)). HTTP PATCH and WebSocket are used to send the changes over the network.

Together, with this protocol, the server 'owns' the view-model and dictates its content. The clients merely suggest changes by sending user input (using PATCHes). In other words, server becomes the "Puppeteer" and the clients become its "Puppets".

The protocol is defined in [Palindrom wiki](https://github.com/Palindrom/Palindrom/wiki/Server-communication).

## Palindrom Implementation

To make it super easy to use this pattern in your web apps, we are using a library called [Palindrom](https://github.com/Palindrom/Palindrom). It is possible to use Palindrom implicitly in your web apps, so using it becomes self-configurable and as easy as putting few lines of code in your app.

For the maximum benefit, use a two-way data binding framework such as Polymer or AngularJS to render your UI in HTML without writing a single line of JavaScript. If you have needs for client-side coding, a JavaScript object is exposed and can be used with any library, such as D3 or React.

In most of the code samples we stay on the bleeding edge of web development by using Palindrom with Polymer and Web Components.


# Client-Side Stack

## Introduction

Apart from data sync with Palindrom, we propose the complete stack for maximum benefit of building thin client SPAs. The other important parts are: a template engine (with two-way data binding) and a template loading infrastructure.

This stack lets you stay on the bleeding edge of development, using the new APIs such as Web Components that turn the web browser into a powerful, interactive presentation layer for apps.

All the client side libraries mentioned on this page come preinstalled with Starcounter. Starcounter auto-configures this stack for you, so don't bother reading this page unless your are interested in the internals.

The stack attempts to add as few new components as possible. The existing established patterns and web standards are used wherever possible.

## Client-Side Libraries

### Data Sync

Palindrom web apps use [Palindrom](https://github.com/Palindrom/Palindrom) for server-client data synchronization. Palindrom makes use of the JSON and [JSON-Patch](https://tools.ietf.org/html/rfc6902) web standards. It uses the JSON,  `XMLHttpRequest` and WebSocket APIs that are built into every modern web browser.

The implementation of JSON-Patch is provided by `json-patch-duplex.js`, which comes from the [fast-json-patch](https://github.com/Starcounter-Jack/JSON-Patch) library and provides tools for applying and generating patches.

The implementation of operational transformation in Palindrom is provided by the following libraries: [json-patch-queue](https://github.com/Palindrom/JSON-Patch-Queue), [json-patch-ot-agent](https://github.com/Palindrom/JSON-Patch-OT-agent), [json-patch-ot](https://github.com/Palindrom/JSON-Patch-OT).

To make it easy to use, Palindrom including all the dependencies is wrapped into a helper Custom Element [palindrom-polymer-client](https://github.com/Palindrom/palindrom-polymer-client). Adding this Custom Element into the DOM automatically loads and configures all the dependencies.

### Template Engine

Since Palindrom provides a JavaScript object that reflects the server-side view-model, it needs a template engine to present the UI in web browser DOM. JavaScript libraries, such as D3 or React, can consume this object.

\
To render HTML templates in the DOM we recommend the [`dom-bind`](https://www.polymer-project.org/1.0/docs/devguide/data-binding) Custom Element that is part of the [Polymer](https://github.com/Polymer/polymer) library. It supports binding annotations using curly braces (`{{}}`), conditional fragments (using [`dom-if`](https://www.polymer-project.org/1.0/docs/devguide/templates#dom-if)) and loops (using [`dom-repeat`](https://www.polymer-project.org/1.0/docs/devguide/templates#dom-repeat)).

Our solution is based on the Web Components APIs from the latest specs for bleeding edge HTML. It's approved by all major browser vendors and is [widely adopted](https://www.webcomponents.org/). However, not everything is there yet, some browsers will still require the `webcomponents.js` polyfill to be loaded to support for these APIs.

### Template Loading

To build modular apps out of many partial views, use the `imported-template` Custom Element. It loads the templates from separate HTML files and stamp them to the DOM, it plays well with Polymer's data-bindings and regular DOM APIs.

To make it easier to use with Starcounter, `importet-template` is wrapped in the [starcounter-include](https://github.com/Starcounter/starcounter-include) Custom Element, which also sets up the data binding between Palindrom and imported-template.

## The Stack, Visualized

The following chart shows the bird-eye's view on the client side libraries used.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe4d83540-8680-11e7-9944-1f85270462c6%2Fclient-side-components.svg?generation=1503327411652753\&alt=media)

## Versions

See the file `ClientFiles\bower-list.txt` in your Starcounter installation directory (usually `C:\Program Files\Starcounter`) for the list of client side libraries bundled with your Starcounter instance, including their version numbers.


# Sessions

## Introduction

Sessions are used to retain the state in your app. A session is represented by an instance of a `Session` class. Together with an instance of a `Json` object it can be used to enable client-server communication and synchronization using JSON patch.

## Session Instance

A new session is created by calling the constructor on `Session` class. To set the new session as active session, it's necessary to assign it to the static property `Session.Current`.

When multiple apps are running, only one of them needs to create the `Session` object and set is as active, because the session identifies the browser tab, not only your app in that tab. Therefore, you always need to check before creating a new session if `Session.Current` is `null`.

{% hint style="info" %}
Each browser tab is a separate state of the UI. Therefore, each tab is tied to its own session. This makes it totally different from the session concept in frameworks like ASP.NET or Zend, where a session stores data from all the browser tabs.
{% endhint %}

Starting with Starcounter 2.3.1.6839 there is a static method, `Session.Ensure()`, that can be used as simplification of the pattern described above that does this check and makes sure that a session with options enabled for patch-versioning and namespaces is set (if not already set) as current and returned. This method can be used everywhere where a session should be created if `Session.Current` is `null`.

```
DateTime createdDate = Session.Ensure().Created; // Session.Ensure() will never be null.
```

Once you have the session, you have the possibility to attach state to it. In Starcounter, the state is represented by a `Json`instance (also called a view-model). The `session` contains a storage where any number of `Json` instances can be kept, using a string as key. This storage is separated per app, so each running app has it own section and can only access it's own state.

{% hint style="info" %}
From Starcounter 2.3.1.6839, `Session.Data` have been obsoleted and replaced with `Session.Store`. Also `Json.Session` is obsoleted. Session should be obtained by using `Session.Current` or `Session.Ensure()`.
{% endhint %}

```csharp
Json state1 = new Json();
Json state2 = new Json();

Session session = Session.Ensure();
session.Store["state1"] = state1;
session.Store["state2"] = state2;
...
state1 = session.Store["state1"]
```

{% hint style="info" %}
The storage on the session is *server-side* only. Nothing stored using `Session.Store` will be exposed to a client. This is a change in behavior of using the old obsoleted `Session.Data`. See the next section on how to attach a `Json` to be used for client-server communication
{% endhint %}

## Client-Server Synchronization with JSON Patch

One additional feature when using `Session`, besides keeping state on the server, is that it can enable client-server synchronization using [JSON Patch](http://tools.ietf.org/html/rfc6902). In short it allows the client and server to send deltas instead of the full viewmodel.

When this is used, the client can send http requests using the `PATCH` verb ([HTTP PATCH method](http://tools.ietf.org/html/rfc5789)) or use websocket to send and receive patches.

To enable this, the session needs to know which `Json` instance should be considered the root viewmodel. If the [PartialToStandalone middleware](https://github.com/Starcounter/Docs/blob/2ffb62f69111b62d73cc9493ef5f190250aa8729/guides/network/middleware#partialtostandalonehtmlprovider) is used, the root viewmodel will be automatically assigned to the session based on the `Json`instance returned from a handler.

There is also a way to manually specify which `Json` instance to use as root, `session.SetClientRoot(json)`. This functionality is included as an extension method instead of directly in the session class. The extension-method can be found in the `Starcounter.XSON.Advanced` namespace.

{% hint style="info" %}
From Starcounter 2.3.1.6839 the handling of determining root viewmodel on session have changed. `Session.PublicViewmodel` and `Session.Data` have been obsoleted and should no longer be used to set client root. Instead use the information in the section above.
{% endhint %}

The whole root viewmodel can be obtained on the client using HTTP GET verb with a url that is sent in the `Location` header for the response of the request that created the session. The location contains a specific identifier for the session and viewmodel that is calculated for each session to be non-deterministic.

## Session Properties

The `Session` object exposes a few useful properties, including:

| Property     | Explanation                                                              |
| ------------ | ------------------------------------------------------------------------ |
| `Created`    | Session creation time (UTC).                                             |
| `LastActive` | Session last active (a receive or send occurred on a session) time (UTC) |

## Sessions Timeout

Inactive sessions (that do not receive or send anything) are automatically timed out and destroyed. Default sessions timeout (with 1 minute resolution) is set in database config: `DefaultSessionTimeoutMinutes`. Default timeout is 20 minutes. Each session can be assigned an individual timeout, in minutes, by setting `TimeoutMinutes` property.

## Session Destruction Callback

User can specify an event that should be called whenever session is destroyed. Session destruction can occur, for example, when inactive session is timed out, or when session `Destroy` method is called explicitly. User specified destroy events can be added using `Session.AddDestroyDelegate` on a specific session.

Session can be checked for being active by using `IsAlive` method.

## Operating on Multiple Sessions

Session is created on current Starcounter scheduler and should be operated only on that scheduler. That's why one should never store session objects statically (same as one shouldn't store SQL enumerators statically) or use session objects in multithreaded/asynchronous programming. In order to save session and utilize it later please use `Session.SessionId`described below.

One can store sessions by obtaining session ID string (`Session.SessionId`). Session strings can be grouped using desired principles, for example when one wants to broadcasts different messages on different session groups. When the session represented by the string should be used, one should call `Session.RunTask(String sessionId, Session.SessionTask task)`. This procedure takes care of executing action that uses session on the scheduler where the session was originally created. This procedure underneath uses `Scheduling.RunTask` thereby it can be executed from arbitrary .NET thread.

There is a variation of `Session.RunTask` that takes care of sessions grouped by some principle: `Session.RunTask(IEnumerable<String> sessionIds, Session.SessionTask task)`. Use it if you want to operate on a group of sessions, like in the following chat app example:

```csharp
[Database]
public class SavedSession
{
   public string SessionId { get; set; }
   public string GroupName { get; set; }
}

Session.RunTask(Db.SQL<SavedSession>("SELECT s FROM GroupedSession s WHERE s.GroupName = ?", myGroupName).Select(x => x.SessionId).ToList(), (Session session, string sessionId) =>
{
  var master = session.Data as MasterPage;

  if (master != null && master.CurrentPage is ChatGroupPage)
  {
    ChatGroupPage page = (ChatGroupPage)master.CurrentPage;

    if (page.Data.Equals(this.Data))
    {
      if (page.ChatMessagePages.Count >= maxMsgs)
      {
      	page.ChatMessagePages.RemoveAt(0);
      }
    page.ChatMessagePages.Add(Self.GET<Json>("/chatter/partials/chatmessages/" + ChatMessageKey));
    session.CalculatePatchAndPushOnWebSocket();
    }
  }
});
```

To schedule tasks on all active sessions, then `Session.RunTaskForAll` should be used (note that it runs on all active sessions and if you only need to update few - use `Session.RunTask`).

## Transmission of the Session Identity

Current session is determined and set automatically before user handler is called.

Starcounter Gateway uses one of the following ways to determine the session that should be used for calling user handler.

* `Location` + `X-Referer` or `Referer` headers: Using HTTP protocol, when creating a new session, the response will contain the `Location` HTTP header with the value of a newly created session. Client code later can extract this session value from the received `Location` header and use the session in subsequent requests, using either of the specified ways. Often `X-Referer` or `Referer` request headers are used to specify the session value.
* Session as handler URI parameter: Session value can be specified as one of URI parameters when defining a handler, for example:

```csharp
Handle.GET("/usesession/{?}", (Session session, Request request) =>
{
    // Implementation
});
```

* Session Cookie:

Use of automatic session cookie and the property `UseSessionCookie` have been obsoleted. Instead enable adding a header on outgoing response by setting property `UseSessionHeader` to `true` and optionally specify name of header with `SessionHeaderName` (default `X-Location`).

The priorities for session determination, for incoming requests, are the following (latter has higher priority than previous): session on socket, `Referer` header, `X-Referer` header, session URI parameter.

## Session Options

The `Session` constructor has an overload that takes the enum `SessionOptions`. This enum has five options:

| Option                 | Explanation                                                                                                                                                                                                                              |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Default`              | Is the default behavior of `Session`, declaring `new Session(SessionOptions.Default)` is the same as using the default constructor.                                                                                                      |
| `IncludeSchema`        | Was added for Starcounter 1.x and does not serve a purpose anymore. Is the same as using the default constructor.                                                                                                                        |
| `PatchVersioning`      | Enables operational transformation with Palindrom. Thus, `PatchVersioning` is required for communication with Palindrom.                                                                                                                 |
| `StrictPatchRejection` | Throws an error instead of rejecting changes in two cases: (1) when an incoming patch tries to access an object or item in an array that is no longer valid and (2) when the client sends a patch with a different format than expected. |
| `IncludeNamespaces`    | Enables namespacing of Typed JSON responses. This is the default behavior and is thus the same as using the default constructor.                                                                                                         |

## CalculatePatchAndPushOnWebSocket

JSON patches are calculated whenever there's an incoming request that asks for JSON patch. This means asynchronous changes to the view-model that are finished after the response is sent will not be included in the response patch.

For example, if a user clicks a button to send an order, the user expects to see a confirmation message when the order has been sent. Since the operation is scheduled on a separate thread that executes asynchronously to avoid blocking, the task to send the order will finish after Starcounter has sent the response. This means that the user will not see the status message until they send another patch that triggers a response that includes the status message.

```csharp
void Handle(Input.SendOrderTrigger _)
{
    Session.RunTask(Session.Current.SessionId, (session, id) =>
    {
        SendOrder(Order);
        Order.StatusMessage = "Done";
    }); // The user will not see the status message until they send another patch
}
```

To fix this, use `CalculatePatchAndPushOnWebSocket`. That lets the user see the status message immediately after the order has been sent.&#x20;

```csharp
void Handle(Input.SendOrderTrigger _)
{
    Session.RunTask(Session.Current.SessionId, (session, id) =>
    {
        SendOrder(Order);
        Order.StatusMessage = "Done";
        session.CalculatePatchAndPushOnWebSocket();
    });
}
```

When calling `CalculatePatchAndPushOnWebSocket`, Starcounter traverses the JSON tree and calculates the difference between the current and previous tree. It then sends patches to the client with the changes.


# HTML Views

## Introduction

This page explains what HTML views and view-models are and how they are used to create nested pages.

## HTML View Definition

An HTML view definition is a valid HTML document that contains at least one `<template>` tag and can be loaded with the [`<starcounter-include>`](https://github.com/Starcounter/starcounter-include) or [`imported-template`](https://github.com/Juicy/imported-template) web component. The `<template>` tag is cached on the client and stamped out as many times as needed.

{% hint style="info" %}
Read about the difference between `starcounter-include` and `imported-template` in the article "[\<starcounter-include> and non-namespaced partial view-models](https://starcounter.io/starcounter-include-non-namespaced-partial-view-models/)".
{% endhint %}

The requirements and behaviors for the HTML view definition originate from the [HTML Imports](http://www.w3.org/TR/html-imports/) and [HTML Template](http://www.w3.org/TR/html-templates/) specifications. The most interesting aspects are:

* The document should be a valid HTML document and can contain anything that the browser allows
* The document may include dependencies such as stylesheets, HTML imports, scripts, etc.
* The content outside of the `<template>` tag is requested and executed once per session on partial load time
* The content inside the `<template>` is stamped out with the `<template>` itself every time the response is bound to a tree in the view-model
* Every node from the `<template>` will be stamped with an attached `model` property. Custom elements, template binding frameworks, or JavaScript scripts can use this data to populate the view

A partial HTML view is a view that has been nested as a part of a bigger view.

{% hint style="warning" %}
The Polymer version used by Starcounter was changed from Polymer 1 to Polymer 2 in Starcounter 2.4. This change requires certain aspects of the view to be adapted to work in Starcounter 2.4. Read more in this blog post: <https://starcounter.io/starcounter-2-4-gets-upgrade-polymer-2/>
{% endhint %}

## View-Model Definition

A view-model definition is a valid JSON document with the filename extension `.json` that contains an `Html` property that points to the corresponding HTML view. It will also contain properties for the data that will be bound between the view and the database model.

The view-model definition can be combined with an optional view-model code-behind which defines the class name and contains input handlers. The filename extension for this file is `.json.cs`.

A partial view-model is a view-model that has been nested as a part of a bigger view-model.

A partial view-model together with its corresponding partial HTML view can simply be referred to as a partial.

## Using Partials

A partial can be accessed in two different ways:

1. Directly through an HTTP request that is handled in the application by a `Handle.GET`. The response to this request should be a complete HTML and JSON document.&#x20;
2. Through a blending point. A blending point is a combination of a `Self.GET` call, which acts as the partial view-model blending point, and a `<starcounter-include>` tag, which acts as a partial HTML view blending point. The `<starcounter-include>` tag determines where in the DOM tree the partial HTML view should be rendered.

## Examples

### Creating the Handler

No matter if the partial is accessed through an HTTP request from the browser or a `Self.GET` call, there always needs to be a handler that deals with the request. This handler returns a partial with the JSON and HTML. The handler, in its simplest form, looks something like this:

```csharp
Handle.GET("/your/partial/url", () => 
{
    return new YourPartialPage();
}
```

This handler only returns the JSON and HTML if the app uses the `HtmlFromJsonProvider` and `PartialToStandaloneHtmlProvider` [middmileware](/2.3.1/guides/network/middleware).

### Adding the Blending Point

You can now create a blending point for this partial by attaching the partial to a parent partial using `Self.GET`. For example:

```csharp
mainPage.SubPage = Self.GET("/your/partial/url");
```

In the partial HTML view for the `mainPage` above, the HTML from the `SubPage` partial can be stamped in like so:

```markup
<starcounter-include partial="{{model.SubPage}}"></starcounter-include>
```

## Partial HTML View Example

A partial HTML view may look something like this:

{% code title="AddressPage.html" %}

```markup
<!--
Load you dependencies: <script>s, HTML Imports, CSS stylesheets etc.
Those dependencies will only be executed once when the partial is imported
-->
<!-- For example, to use Polymer's dom-bind custom element: -->
<link rel="import" href="/sys/polymer/polymer.html" />
<style>
    .myapp-address-entry-name {
        font-weight: bold;
    }
</style>
<!--
Everything from this template will be stamped into the parent page's DOM and <script>s will be executed for every instance of this partial.
All child nodes will receive a `model` property with JSON view-model,
which is automatically bound to the server-side.
-->
<template>
    <template is="dom-bind">
        <!--
            Now, the double curly brace syntax "{{}}" can be used 
            for two-way data bindings from the HTML view to serverside. 
            For example:
         -->
        <h2 class="myapp-address-entry-name">{{model.FullName}}</h2>
        <h4>Address</h4>
        <starcounter-include partial="{{model.Address}}">
        </starcounter-include>
    </template>
</template>
```

{% endcode %}

The example above uses Polymer's `dom-bind` element for data-bindings. Using Polymer in HTML views is not required, but we prefer it because it is a simple declarative alternative to imperative binding data to HTML elements using JavaScript.


# App Shell

## Introduction

A view exposed from your app should follow the [inversion of control](https://en.wikipedia.org/wiki/Inversion_of_control) principle. It needs to import the dependencies and be able to render itself. It must not assume how and where it is inserted in the main document and must not expect the presence of any APIs other than `<starcounter-include>` and [DOM](https://dom.spec.whatwg.org/#shadow-trees).

The view is inserted in the main document by another view or, if it is at the root of the view-model, by the Starcounter app shell.

{% hint style="info" %}
&#x20;"App shell" is a term coined by Google that is explained well at their web page [The App Shell Model](https://developers.google.com/web/fundamentals/architecture/app-shell).
{% endhint %}

## Starcounter app shell

The Starcounter app shell is a minimalist, configurable HTML document that imports the prerequisites for all blendable web apps:

* &#x20;a polyfill for the browsers that do not have a native implementation of the Web Components specs
* &#x20;Palindrom JS library for communication with the server
* &#x20;the `<starcounter-include>` custom element
* &#x20;Bootstrap, a CSS framework

On page load, the app shell establishes the Palindrom session with the server, obtains the JSON view-model tree, inserts the root view using `<starcounter-include>` and connects the root view to the view-model.

## Obtaining and invoking the app shell

The default content of the app shell is tied to the StarcounterClientFiles version that is bundled with your Starcounter installation. The current version can be previewed in the [StarcounterClientFiles repository on GitHub](https://github.com/Starcounter/StarcounterClientFiles/blob/3.x/src/StarcounterClientFiles/wwwroot/sys/app-shell/app-shell.html). You can upgrade or downgrade the app shell by installing another version of StarcounterClientFiles from the App Warehouse.

The app shell is invoked automatically for any `Json` with a `Html` property that is returned from your app. The only provision is that your app needs to register the following middleware:

```csharp
Application.Current.Use(new HtmlFromJsonProvider());
Application.Current.Use(new PartialToStandaloneHtmlProvider());
```

The [middleware](/2.3.1/guides/network/middleware#middleware-classes) page explains the APIs presented above. In short, the `HtmlFromJsonProvider` middleware fetches the view associated with a view-model, and `PartialToStandaloneHtmlProvider` wraps the view in the app shell.

## Altering the app shell

It is strongly advised to use the default, unmodified app shell. The default app shell is the lowest common denominator for all apps. If you change it, you might introduce side effects that make your app incompatible with other vendor's apps.

If you need to modify the app shell, follow the instructions on the [middleware](/2.3.1/guides/network/middleware#partialtostandalonehtmlprovider) page.<br>


# Web Components

## Introduction

If you are writing web applications and you're allowed to use modern browser features, Web Components is a very powerful ally. A good introduction can be found on [WebComponents.org](https://www.webcomponents.org/introduction/), but we will give you a summary and how to use Web Components in Starcounter web apps.

## Summary of Web Components

Web Components are a set of new features in HTML 5 and beyond. Each feature is useable on its own and is not dependent on the other features, but together they really have the potential to change the way web applications are developed.

For instance, Web Components allow us to render a list of products and their barcodes by executing this code in a web browser:

```markup
<link rel="import" href="/sys/x-barcode/src/x-barcode.html">
<link rel="import" href="/sys/palindrom-client/palindrom-client.html">
<template id="root">
  <template is="dom-repeat" items="{{model.Products}}">
    <h1>{{item.Name}}</h1>
    <x-barcode code="{{item.Barcode}}"></x-barcode>
  </template>
</template>
<palindrom-client ref="root"></palindrom-client>
```

This is only a fraction of what would be required in traditional frameworks, such as Backbone.

### `<template>` Tag

Starcounter let's you create JSON view-models that expose the current application state. To render it in the UI, it is the simplest to use a client-side framework that provides two-way data binding between HTML and JSON.

The `<template>` tag that was added to HTML5 as part of the Web Components spec family. It allows to define reusable chunks of HTML that work with any framework that supports this new web standard. Because it is now an established standard, we decided to use it to power [partials](/2.3.1/guides/web-apps/html-views).

### Custom Elements and Shadow DOM

We suggest to keep the business and application logic on the server. Yet, there is still plenty of room to use the capabilities of a modern web browser to run the *view* logic.

Custom Elements and Shadow DOM define how one can wrap a complex view functionality into a simple HTML tag, that requires no knowledge about the implementation details from the developer.

As can be seen in the above code snippet, to render a UPC-A barcode using SVG in a web browser, you can just import and use a MIT-licensed Custom Element called [x-barcode](https://github.com/girliemac/x-barcode). There is no learning curve and you don't need to care about the implementation details.

Thousands of open sourced Custom Elements can be found on [customelements.io](https://customelements.io).

### HTML Imports

Last but not least, the HTML Imports spec is the part of Web Components family that defines how to obtain fragments of code using a `<link>` tag. As can be seen on the above snippet, this is how a definition of a Custom Element is loaded.

In addition to loading Custom Elements, Starcounter takes benefit from HTML Imports to compose complex apps using small templates known as [partials](/2.3.1/guides/web-apps/html-views), but you don't need to care much about that because it is done behind the scenes.

## Using Web Components

Web Components are loaded by browser with HTML Imports. HTML Import for Polymer in Starcounter should look like this:

```markup
<link rel="import" href="/sys/polymer/polymer.html" />
```

### Avoiding Loading the Same Files Multiple Times <a href="#avoiding-loading-the-same-files-multiple-times" id="avoiding-loading-the-same-files-multiple-times"></a>

Browser loads HTML Imports only once, not like scripts and styles, which are loaded as many times as many references page has.

How does the browser know which of the components has already been imported then? Browser distinguishes Web Components one from another by their location.

This will be loaded twice and then lead to a Polymer error:

```markup
<link rel="import" href="/bower_components/polymer/polymer.html" /><link rel="import" href="/sys/polymer/polymer.html" />
```

It is a good practice to avoid double loading of external libraries. Hence, you need to use the same URL pattern as other apps. The recommended way is to use the pattern: `/sys/<dependency-name>/<dependency-files>`.

Starcounter has system folder called `StaticFiles` and located in the installation folder: `C:\Program Files\Starcounter\ClientFiles\StaticFiles`. There you can find `sys` folder, which includes some common components such as Polymer and Palindrom. You can also look at the code example under `PartialToStandaloneHtmlProvider` on the [middleware page](https://docs.starcounter.io/guides/network/middleware) to see what client side libraries are included in that directory by default.

The benefit of this is that you can rely on having a specific version of Starcounter to include a specific version of Polymer, Palindrom, etc.

### Adding External Dependencies to Your Apps <a href="#adding-external-dependencies-to-your-apps" id="adding-external-dependencies-to-your-apps"></a>

In order to add files that match this pattern, simply put a `sys` folder in your `wwwroot` folder that holds the static files for your project.

You should not do that automatically, but use Bower to install such dependencies. A correct Bower configuration consists of two files in your project: `.bowerrc` and \`bower.json.

#### .bowerrc <a href="#bowerrc" id="bowerrc"></a>

The `.bowerrc` file contains the Bower configuration. It specifies the destination directory and what dependencies should be ignored, because they are delivered with Starcounter. An example of this can be found in the [KitchenSink app](https://github.com/StarcounterApps/KitchenSink/blob/master/src/KitchenSink/.bowerrc).

To find the specific dependencies that are delivered with Starcounter, go to `C:\Program Files\Starcounter\ClientFiles\bower-list.txt`. For Starcounter 2.3.1.6694, it looks like this:

```
bower check-new     Checking for new versions of the project dependencies...
sys#1.0.0 D:\repos\Starcounter\src\BuildSystem\ClientFiles
├─┬ PuppetJs#2.5.2 extraneous (latest is 3.0.4)
│ ├── fast-json-patch#1.2.2 (latest is 2.0.5)
│ ├── json-patch-ot#61992acdfd extraneous
│ ├─┬ json-patch-ot-agent#49d5d2cee5 extraneous
│ │ └── json-patch-queue#2a579f94db
│ └── json-patch-queue#2a579f94db
├── array.observe#0.0.1 extraneous
├─┬ bootswatch#3.3.7 (latest is 4.0.0-alpha.6)
│ └── bootstrap not installed
├── document-register-element#1.7.0
├── dom-bind-notifier#a43453ceb9
├─┬ imported-template#1.5.0
│ └── juicy-html#1.2.0
├── json-patch-ot#61992acdfd extraneous
├─┬ json-patch-ot-agent#49d5d2cee5 extraneous
│ └── json-patch-queue#2a579f94db
├── juicy-redirect#0.4.2 extraneous
├── object.observe#0.2.6 extraneous
├─┬ palindrom-client#4.0.0
│ ├── Palindrom#3.0.3 (3.0.4 available)
│ └── polymer#1.9.3 (latest is 2.0.2)
├── polymer#1.9.3 (latest is 2.0.2)
├─┬ puppet-client#4.0.0
│ ├── Palindrom#3.0.3
│ └─┬ polymer#1.9.3
│   └── webcomponentsjs#0.7.24
├── puppet-redirect#0.4.3 (latest is 0.5.0)
├── shadycss#1.0.5 extraneous
├─┬ starcounter-debug-aid#2.0.11
│ ├─┬ juicy-jsoneditor#1.1.0 (1.2.0 available)
│ │ ├── fast-json-patch#1.2.2 incompatible with ~1.0.0 (1.0.1 available, latest is 2.0.5)
│ │ ├── jsoneditor#5.5.11 (5.9.5 available)
│ │ └── polymer#1.9.3 (2.0.2 available)
│ └── polymer#1.9.3 (2.0.2 available)
├─┬ starcounter-include#3.0.0-rc.3
│ ├── imported-template#1.5.0
│ └── translate-shadowdom#0.0.5
└── webcomponentsjs#0.7.24 (latest is 1.0.10)
```

#### bower.json <a href="#bower.json" id="bower.json"></a>

`bower.json` file that keeps the list of your app's client side dependencies. This file should not be created and maintained manually. It should be modified using the command line tool: `bower init`, `bower install paper-dialog --save`.

A sample file can be found in the [KitchenSink app](https://github.com/StarcounterApps/KitchenSink/blob/master/src/KitchenSink/bower.json).

## Static File Server <a href="#starcounter-static-file-server" id="starcounter-static-file-server"></a>

The `StaticFiles` folder from Starcounter installation is automatically served as a static content folder. When Starcounter server receives a request for a static file, it searches for the file in all of the static content folders. The project folder has higher priority over internal folder.

Read more [here](https://docs.starcounter.io/guides/network/static-file-server).

Keep that in mind you can simply use another version of Polymer by putting it into your local `sys` folder. This will affect all other apps, though.


# View Attaching

## Introduction

When the browser sends a request to the server, the app with the matching URI sends back a response with the view. Starcounter has a unique feature of *attaching* views from other apps to the main view in the same response using a system based on *tokens* and *contexts*.

For example, if a user wants to see a profile of a person, the browser makes a request to the People app: `GET http://localhost:8080/people/person/4782`. The response that is sent to the browser will include not only the view from the People app but also views from other apps that are attached to it.

On a high level, these are the specific steps involved with sending the response:

1. The browser makes a request for a resource, such as:  `GET http://localhost:8080/people/person/4782`.
2. The server receives the request and routes it to the corresponding handler in an app (People).
3. The handler is mapped to a token that describes the sort of resource it deals with, in this case, it's the  person table in the database (`simplified.ring2.person`).&#x20;
4. The server requests responses from all the other handlers registered in the code host that are mapped to the same token.
5. For the responses from other apps that are views, the server attaches the views to the view from the initial handler (which is called the main view) during the serialization process.
6. The server sends the response, which now contains views from multiple apps, back to the client.

By using tokens, the apps don't need to know anything about other apps in the code host - they don't even need to know if there are other apps -  they only have to communicate what concept the handlers deal with. Thus, apps should be built to not depend on, or expect, attaching.&#x20;

The process of mapping handlers to tokens and attaching responses is handled by the `Blender` class in the `Starcounter` namespace.

Attaching works well with [view composing](broken://pages/-KtAtBc3h5rvvIV8gBg1) to make different apps looks like one.

{% hint style="info" %}
Attaching was previously called "server-side blending"
{% endhint %}

## Tokens

Attaching uses tokens. These tokens are either strings or classes. Handlers with the same token are called on internal `Self.GET` calls or external URI that matches one of the handlers. Once the handler with a token is called, it will not trigger further calls mapped to that handler directly, only when a new `Self` call is made.

The first parameter is either a handler URI or a specific URI. If the token is a string, it's defined as the second parameter. If it's one or more classes, they are defined in the template or as a `Type` array parameter.

```csharp
Blender.MapUri("/Products/settings", "settings");
Blender.MapUri<Product>("/Products/partials/product/{?}");
Blender.MapUri("/Products/menu", "menu");
Blender.MapUri("/people/persons/34623", "person"); // Specific URI blending.
Blender.MapUri<Person>("/people/persons/{34623}"); // Specific URI blending using so-called mixed URI.
```

An arbitrary number of classes are allowed as tokens (up to 3 in template, more in array of class `Type`). Here are the `MapUri` signature examples (same exist for removing token mapping from a handler):

```csharp
static void MapUri<T>(String uri, String[] contexts = null);
static void MapUri<T1, T2, T3>(String uri, String[] contexts = null);
static void MapUri(String uri, Type[] types, String[] contexts = null);
```

Handlers can be mapped to empty tokens:

```csharp
static void MapUri(String uri, String[] contexts = null);
static void MapUri(String uri, Boolean allowFromUriConverter, Boolean allowToUriConverter, String[] contexts = null);
```

Handlers with empty tokens are called with other handlers with the same empty token.

## Contexts

Token matching can be made more fine-grained by using contexts. They are composed of a list of strings that acts as a bit map when matched with other contexts. No context (`null` value) means **match any context**. Otherwise, two handlers are matched if source context contains same elements as destination context. Examples:

* Source context `{ "Readable", "View" }` is NOT matched with `{ "Writable", "View" }`.
* Source context `null` is matched with `{ "Readable", "View" }` and `{ "Writable", "View" }` and any other context.
* Source context `{ "View", "Writable" }` is matched with `{ "Writable", "View" }` and vice versa.

Consider contexts as an additional matching rule for handlers with the same token.

### Attaching Specific URIs

&#x20;Specific URI is the handler URI with parameters supplied. For example, for the handler `/people/{?}` the specific URIs will be `/people/john`, `/people/bob`, etc. When calling `Blender.MapUri` for a specific URI, you should pass a mixed URI which indicates what handler it belongs to. For the previous examples, the mixed URI will be `/people/{john}` and `/people/{bob}`, so the parameter in specific URI is wrapped into curly brackets. `Blender` class has helper methods to construct such mixed URIs: `Blender.GetMixedUriFromHandlerAndParameters`, `Blender.TryGetMixedUriFromSpecific`. The last method tries to find corresponding handler for the given specific URI, which might not be determined correctly (for example, in case when the corresponding handler is not yet registered). Mixed URIs in `MapUri` calls are needed so the underlying handler for a specific URI can be identified. Other `Blender` methods like `UnmapUri`, `IsMapped`, etc. can still use specific URIs and not mixed.

## Dynamic Addition and Removal

Handlers that are registered and mapped to a token can dynamically have the mapping removed with `UnmapUri`:

```csharp
Blender.UnmapUri("/app4/{?}", token2);
Blender.UnmapUri("/twoparams3/{?}/{?}", token);
Blender.UnmapUri<MyClass>("/app1/{?}");
Blender.UnmapUri<Person>("/people/persons/34623"); // Unmapping specific URI (not that specific URI is used, not mixed).
Blender.UnmapUriForAllTokens("/app1/param1/{?}"); // Unmapping URI for all tokens.
```

`IsMapped` is used to check if a handler is mapped to a token:

```csharp
isInBlender = Blender.IsMapped("/noparam1", token3);
isInBlender = Blender.IsMapped<MyClass>("/noparam1");
isInBlender = Blender.IsMapped<Person>("/people/persons/34623"); // Checking if specific URI is blended.
```

To get a dictionary of all handlers and their respective tokens, use `ListAllByTokens` or `ListAllByUris`:

```csharp
static Dictionary<string, List<BlendingInfo>> ListAllByTokens();
static Dictionary<string, List<BlendingInfo>> ListAllByUris();
```

The dictionary returned by `ListAllByTokens` has the token as the key. The entries in the dictionary are sorted alphabetically based on the token. The same applies for `ListByAllUris` but for URIs.

To get the list of blending candidates that are going to run for a given URI handler:

```csharp
BlendingInfo[] GetRunCandidatesForUri(String uri);
```

To list registered blending rules for a given URI:

```csharp
BlendingInfo[] ListByUri(String uri)
```

and the same but for a given token:

```csharp
BlendingInfo[] ListByToken(String token)
```

As you might noticed, a special blending information structure is used here: `BlendingInfo`.\
It contains the following methods/properties:

```csharp
String AppName; // Returns the application name to which the handler belongs.
Boolean IsActive; // Shows if this blending is active.
String Token; // Returns the token.
String Uri; // Returns URI given in MapUri (which was either mixed, specific or handler URI).
String SpecificUri; // Returns a specific blended URI or null if MapUri was called with handler URI.
String HandlerUri; // Returns the handler URI.
Boolean IsSpecificUri; // Is it a specific URI attached, not a handler URI.
String[] Contexts; // Returns context (if any).
Boolean IsFromUriConverterOn; // Returns True if the "FromUriConverter" is set/allowed.
Boolean IsToUriConverterOn; // Returns True if the "ToUriConverter" is set/allowed.
```

## Parameters in Handlers

Handlers are allowed to have an arbitrary number of parameters. When there is at least one parameter, the conversion functions are used. First converter translates handler arguments to token arguments, while the second converter does the opposite. Both converters are taking and returning an array of strings:

```csharp
Blender.MapUri("/twoparams1/{?}/{?}", token,
(String[] from) => {
   return from;
}, (String[] to) => {
   return to;
});
```

In the example above, converter passes handler parameters to token parameters, as is. There are signatures of `Blender.MapUri` with no converters, which means that parameters are simply passed through, like above.

Often it's needed to trigger attachment on a specific URI. To achieve this, first converter should return non-null string array on certain parameters.

## Call Direction

You can specify the direction of which handlers should be called. This is needed to trigger attaching in a certain direction: from handler or to handler. The direction is determined by the value that corresponding converter is returning: `null` converter or `null` string array returned in converter blocks the direction of the call. In case of zero parameters, there is a special `Blender.MapUri` override with corresponding boolean parameters to determine the allowed call directions.

```csharp
Blender.MapUri("/twoparams1/{?}/{?}", myToken, null, (String[] to) => {
    return to;
});
Blender.MapUri("/noparam1", token3, true, false);
```

In the first example above, handler "/twoparams1/{?}/{?}" should not trigger calls on the `myToken` token, but it can be called when attaching is triggered by other handlers on the same token.\
In the second example above, handler "/noparam1" can trigger other handlers on the same token, but can't be triggered by them.

## Same Handler - Different Tokens

Handlers can be mapped to multiple tokens:

```csharp
Blender.MapUri("/app1/{?}", token1);
Blender.MapUri("/app2/{?}", token1);
Blender.MapUri("/app3/{?}", token1);

Blender.MapUri("/app1/{?}", token2);
Blender.MapUri("/app4/{?}", token2);
```

Here, the handler `/app1/{?}` is mapped to the `token1` and `token2`.

When an app requests `Self.GET("/app1/xyz")`, both handlers mapped to `token1` and `token2` will be triggered.

## Separate blending rules in JSON

You can describe blending rules in JSON, separate from the application code. If the file `blend.json` is in the same directory as the starting application assembly - the file is parsed and the extracted blending rules are applied right after the `Main` method. The same happens when the application restarts. With this mechanism, blending rules can be changed for shipped application, where you only have the binaries. Here is an example of the content of a `blend.json` file:

```javascript
[
  {
	 "Uri": "/someuri1",
	 "Token": "sometoken1",
	 "Contexts":[],
	 "AllowFromDirection": true,
	 "AllowToDirection": false
  },
  {
	 "Uri": "/someuri2/{?}/bla",
	 "Token": "",
	 "Contexts":["context1","context2"],
	 "AllowFromDirection": false,
	 "AllowToDirection": true
  },
  {
	 "Uri": "/someuri3/{name}/xxx",
	 "Token": "sometoken3",
	 "Contexts":["context1","context2", "context3"],
	 "AllowFromDirection": true,
	 "AllowToDirection": true
  }
]
```

Each element of the array contains the following fields:

```csharp
string Uri; // Mixed, specific or handler URI.
string Token; // Blending token.
string[] Contexts; // Blending contexts (if any).
bool AllowFromDirection; // Allows blending calls from this URI.
bool AllowToDirection; // Allows blending calls to this URI.
```

In short, the meaning of the above fields is the same as in `Blender.MapUri` functions variations.


# View Composing

## Introduction

When [Attaching views](/2.3.1/guides/web-apps/view-attaching) from different apps, the views are stacked on top of each other. In most cases, that's not what we want. Instead, we would like to compose elements in the views to make the result look like one page. That's what View Composing does:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2F-KxJ42Q8HA7mxEGNRQ4t%2F-KxJ4448qoxBz08gGzOp%2Fassets%252Fstarcounter%252Fe48b74d0-8680-11e7-9944-1f85270462c6%252Fe5df1df0-8680-11e7-9944-1f85270462c6%252FSimpleBlendingDemo.png?alt=media\&token=3f17deb4-6fdf-44af-b7b4-2d136d18b401)

In the example above, there are two apps, PetList and MedicalRecord. By default, they are stacked on top of each other. This makes it seem like they are not related, when they actually are. By Composing, we can move the table of examinations into the card from the PetList app and make it look like one coherent concept. In essence, we are **changing the composition but not the content** to combine apps that were not explicitly built to share the same screen. This is done without touching the source code of the individual apps.

This ability of modifying the composition of views coming from different apps is crucial, especially when working with many apps. Without it, there would just be a stack of views with no meaningful visual context, as shown in this illustration:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2F-KxJ3HgBswFUfeLQo-NY%2F-KxJ3JRMTnyEgZ2UpLF8%2Fassets%252Fstarcounter%252Fe48b74d0-8680-11e7-9944-1f85270462c6%252Fe5df1df1-8680-11e7-9944-1f85270462c6%252Fclientsideblending.png?alt=media\&token=6c34809a-9d43-474b-a278-da5f435539b5)

{% hint style="info" %}
Composing was previously called "client-side-blending"
{% endhint %}

## Composition and Content Separation

Composing works by replacing or modifying the default structure of HTML elements. For this to work, content and composition has to be separated. [Shadow DOM](https://www.html5rocks.com/en/tutorials/webcomponents/shadowdom/) handles this separation - the content is in light DOM and the composition is in Shadow DOM.

The structure of this separation looks like this:

```markup
<template>
    <h1 slot="myapp/main-heading">My heading</h1>
    <button slot="myapp/left-button">Go left</button>
    <button slot="myapp/right-button">Go right</button>
    <template is="declarative-shadow-dom">
        <style>
            .myapp-direction-controls {
                display: flex;
                justify-content: center;
            }
        </style>
        <slot name="myapp/main-heading"></slot>
        <div class="myapp-direction-controls">
            <slot name="myapp/left-button"></slot>
            <slot name="myapp/right-button"></slot>
        </div>
    </template>
</template>
```

Here, the content of the view is defined on the root level and the composition is defined inside the `declarative-shadow-dom`. The [slot](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/slot) elements are insertion points for the content into the composition.

The `declarative-shadow-dom` is used as the default composition that can be further modified or replaced in runtime using the Starcounter system apps CompositionEditor and CompositionProvider.

## Composing the attached views

### Default compositions of the attached views

With the MedicalProvider and PetList example we have two views, each with its default composition:

```markup
<style>
    @import '/PetList/style.css';
</style>
<div class="pet-list-wrapper">
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-name"></slot>
        <slot name="petlist/details-age-and-animal"></slot>
    </div>
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-owner-name"></slot>
        <slot name="petlist/details-weight"></slot>
    </div>
    <slot name="petlist/details-list-link"></slot>
</div>
```

```markup
<slot name="medicalrecord/records-list-headline"></slot>
<slot name="medicalrecord/records-list-table"></slot>
```

When these two views are attached, the default composition from MedicalRecord is appended at the end of the PetList wrapper. The resulting composition looks like this:

```markup
<style>
    @import '/PetList/style.css';
</style>
<div class="pet-list-wrapper">
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-name"></slot>
        <slot name="petlist/details-age-and-animal"></slot>
    </div>
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-owner-name"></slot>
        <slot name="petlist/details-weight"></slot>
    </div>
    <slot name="petlist/details-list-link"></slot>
</div>
<slot name="medicalrecord/records-list-headline"></slot>
<slot name="medicalrecord/records-list-table"></slot>
```

### Custom compositions of the attached views

To create a custom composition, we will move the MedicalRecord table and headline into the `div class="pet-list-wrapper"` and expand the width of the wrapper to fit the table:

```markup
<style>
    @import '/PetList/style.css';
    .pet-list-wrapper {
        max-width: 750px;
    }
</style>
<div class="pet-list-wrapper">
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-name"></slot>
        <slot name="petlist/details-age-and-animal"></slot>
    </div>
    <div class="pet-list-wrapper__row">
        <slot name="petlist/details-owner-name"></slot>
        <slot name="petlist/details-weight"></slot>
    </div>
    <slot name="medicalrecord/records-list-headline"></slot>
    <slot name="medicalrecord/records-list-table"></slot>
    <slot name="petlist/details-list-link"></slot>
</div>
```

We have now produced the result shown in the image above; the view from the MedicalRecord app has been neatly integrated with the view from PetList.

The tool for creating these custom compositions is the [CompositionEditor](https://github.com/starcounterapps/CompositionEditor) app. When this app is running, you can open it at any page with `Ctrl + E`. When the app opens, it gives you an HTML editor for the composition of your attached views.&#x20;

There are two main concepts in the CompositionEditor - the identifier and composition ("layout" on the below picture).

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5df4500-8680-11e7-9944-1f85270462c6%2Fcompositioneditor.png?generation=1503327413576145\&alt=media)

The identifier represents the unique set of the attached views. For example, the identifier for the PetList and MedicalProvider example looks like this:

```
[partial-id="/sc/htmlmerger?PetList=/PetList/views/PetDetails.html&MedicalRecordProvider=/MedicalRecordProvider/views/RecordsList.html"]
```

The `/sc/htmlmerger` is a prefix that is added every time a view contains views that come from more than one response. In the identifier above, it's the merged views of `PetDetails.html` and `RecordsList.html`.

The composition changes made in the editor are displayed in real time.

Compositions are saved in the database and can be queried for with `SELECT * FROM Starcounter.HTMLComposition`. Since the CompositionEditor interacts with the database, it should not run in production.\
\
Compositions can also be accessed with the static methods `GetUsingkey(string key)` and `GetUsingKeyAndVersion(string key, string version)`on the `HTMLComposition` class. These methods return an `HTMLComposition` with the specified key, or key and version.

### Providing custom compositions

Custom compositions are provided by the [CompositionProvider](https://github.com/starcounterapps/compositionprovider) app. When views are merged, this app checks if there's a composition in the database with the same identifier and serves it, otherwise, the default composition is used.

Because of this, the CompositionProvider has to be running if you want to render custom compositions of your views.

## Result

![Composing in action](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2F-Kwf--lq0R0fTdRgSluS%2F-Kwf-C7Gp_OhUgDkxToZ%2Finside.PNG?alt=media\&token=315bed05-e74e-4a2a-83a9-10b17685ee84)

This screenshot is a result of several of our sample apps running together:

* [Website](https://github.com/Starcounter/Website)
* [People](https://github.com/StarcounterApps/People)
* [Images](https://github.com/StarcounterApps/Images)
* [SignIn](https://github.com/StarcounterApps/SignIn)
* [Search](https://github.com/StarcounterApps/Search)

As you can see, they look like one app because of Attaching and Composing.

## Summary

Composing allows us to make views coming from multiple apps to look like one by rearranging the HTML elements the Shadow DOM. Thus, the actual content and functionality defined in the light DOM will stay the same. The revised composition is stored in the database and retrieved when the views are attached. All of this is done without touching the source code of the apps.

## Read More

* [Layout compositions for HTML partials](https://starcounter.io/layout-compositions-html-partials/) - Blog October 2016
* [Html Views Blending Guidelines](/2.3.1/guides/web-apps/html-view-guidelines) - Docs
* [Shadow DOM v1: Self-Contained Web Components](https://developers.google.com/web/fundamentals/getting-started/primers/shadowdom) - Google Developers


# HTML Compositions

## Introduction

When you want to deploy your custom HTML composition, you have two options:

* Use the REST API for HTML compositions
* Use the [CompositionEditor](https://github.com/starcounterapps/compositioneditor)

## REST API for HTML Compositions

{% hint style="warning" %}
The HTML composition REST API is deprecated from Starcounter 2.3.1. Compositions can still be accessed through the class `HTMLComposition`
{% endhint %}

The REST API allows importing compositions for sets of blended HTML partials. The CompositionEditor has to run for this API to work.

## Composition

<mark style="color:blue;">`GET`</mark> `http://localhost:8080/sc/partial/composition?key={?}&ver={?}`

Returns the HTML of a specific composition

#### Query Parameters

| Name | Type   | Description                     |
| ---- | ------ | ------------------------------- |
| key  | string | The key of the composition.     |
| ver  | string | The version of the composition. |

{% tabs %}
{% tab title="200 The HTML of the requested composition" %}

```javascript
<style>:host{display:block}</style>
<style>
    .kitchensink-layout {
        display: flex;
    }

    .kitchensink-layout__column-left {
        flex: 0 0 160px;
        margin-right: 20px;
    }

    .kitchensink-layout__column-right {
        flex: 0 1 600px;
    }
</style>
<div class="kitchensink-layout">
    <nav class="kitchensink-layout__column-left">
        <slot name="kitchensink/nav"></slot>
    </nav>
    <div class="kitchensink-layout__column-right">
        <slot name="kitchensink/current"></slot>
    </div>
</div>
```

{% endtab %}
{% endtabs %}

## Composition

<mark style="color:green;">`POST`</mark> `http://localhost:8080/sc/partial/composition?key={?}&ver={?}`

Creates a new composition at the specified key if there's no composition there. The body consists of the composition HTML

#### Query Parameters

| Name | Type   | Description                     |
| ---- | ------ | ------------------------------- |
| key  | string | The key of the composition.     |
| ver  | string | The version of the composition. |

{% tabs %}
{% tab title="204 " %}

```javascript
```

{% endtab %}
{% endtabs %}

## Composition

<mark style="color:red;">`DELETE`</mark> `http://localhost:8080/sc/partial/composition?key={?}&ver={?}`

Deletes the composition with the specific key and version

#### Query Parameters

| Name | Type   | Description                                                                   |
| ---- | ------ | ----------------------------------------------------------------------------- |
| key  | string | The key of the composition. If the key is "all", all composition are deleted. |
| ver  | string | The version of the composition.                                               |

{% tabs %}
{% tab title="204 " %}

```javascript
```

{% endtab %}
{% endtabs %}


# HTML Views Blending Guidelines

## Introduction

In order to harness the full power of Starcounter, applications should be built to accomodate for complete visual and functional interoperability. To make this process easier for developers, we provide these guidelines for HTML views which, when followed, will allow applications to achieve seamless visual integration with other applications.

To get a technical background, the article [Layout compositions for HTML partials](https://starcounter.io/layout-compositions-html-partials/) covers more of the underlying ideas of what's presented here. It may also be worth to take a look at the following articles:

* [Unobtrusive styling and composing 3rd party HTML content](http://starcounter.io/unobtrusive-styling-composing-3rd-party-html-content/)
* [HTML partials/includes WebComponents-way](http://starcounter.io/html-partialsincludes-webcomponents-way/)

[CompositionProvider](https://github.com/Starcounter/CompositionProvider) has to run for the code in these guidelines to work. There are two ways to start `CompositionProvider`: follow the instructions in the [README file](https://github.com/Starcounter/CompositionProvider/blob/master/README.md), or, when Starcounter is running, go to `http://localhost:8181/#/databases/default/appstore`, click on the download button next to `CompositionProvider`, and click `Start` at `http://localhost:8181/#/databases/yourDatabase`. This requirement is temporary.

## Guideline 1: Separation of Layout and Content

To make applications look great when running independently while also allowing them to visually blend with other applications, it is beneficial to separate the layout and the content. This is accomplished using the `<template is="declarative-shadow-dom">` element.

The basic boilerplate of a Starcounter HTML view, which is created by adding a `Starcounter HTML template with dom-bind` file in Visual Studio, looks like this:

```markup
<link rel="import" href="/sys/polymer/polymer.html">

<template>
    <template is="dom-bind">

    </template>
</template>
```

To separate the layout and content in this file, the element mentioned above, `<template is="declarative-shadow-dom">` should be used. This element should contain the layout of the HTML view while the `<template is="dom-bind">` should contain the content. Note that this only applies when using Polymer as a templating engine. When using other frameworks, it will not use `dom-bind`, although, the principle of separating the layout from the content will stay constant. In code, this is how it looks:

```markup
<link rel="import" href="/sys/polymer/polymer.html">

<template>
    <template is="dom-bind">
        <!-- content goes here -->
    </template>
    <template is="declarative-shadow-dom">
        <!-- layout goes here-->
    </template>
</template>
```

The content are the elements that either contain information for the user, such as `<h1>`, `<span>`, and `<a>`, or elements that create some kind of interaction between the user and the application, such as `<button>` and `<input>`.

Keep in mind that the only elements that have to be in the `<template is="dom-bind">` are the ones that use Polymer bindings, as denoted with double curly-bracket syntax: `{{model.SomeProperty}}`.

The content of the HTML view is distributed using a Shadow DOM concept called slots, as explained in guideline 4.

## Guideline 2: Defining the Content

When defining the content of a view, it is important to keep in mind that the slotable elements, which are the ones that will be exposed for blending, have to be on the root of the HTML view. Consider the following HTML view:

```markup
<template>
    <h1>A Headline</h1>
    <p>Some text for te user to read</p>
    <button>A button to click</button>
</template>
```

Here, every element is at the root of the document and will be exposed for blending after they are attached to slots and distributed in the Shadow DOM. In some cases, putting every element on the root of the view, like we do in the example above, might not be desired. Then, the goal should still be to place as many elements as possible on the root of the view, especially elements like `<img>`, `<table>`, and custom elements that are visually obtrusive and might require blending to create a high level of interoperability.

Additionally, there might be situations where the developer would like to have a higher level of abstraction on some of his or her content. For example, consider this pagination bar:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2F-Kv92T-0M2mcNmMFTzGQ%2F-Kv92tr7n_MyX9xycoOn%2Fpagination.PNG?alt=media\&token=66584acf-53b1-4637-9157-03d01f917bf2)

Here, it would not make sense to break it up into the respective parts because they do not have any real meaning when presented individually. It would rather make sense to put the parent on the root level so that the whole bar is exposed for blending, and not the individual buttons.

## Guideline 3: Attaching the Content to Slots

To have better flexibility in blending - to be able to distribute an element individually - the element needs to be assigned to [named slots](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/slot#named-slot), like `<button slot="myapp/submitbutton">Submit</button>`. Such an element could be distributed via `<slot name="myapp/submitbutton"></slot>` in the Shadow DOM. Prefix slot names with the app name, to avoid collisions.

When no slot name is provided for an element it will be distributed in [the default slot](https://w3c.github.io/webcomponents/spec/shadow/#shadow-tree-slots): `<slot></slot>`.

{% hint style="info" %}
&#x20;The first slot in a shadow tree, in tree order, whose name is the empty string, is sometimes known as the "default slot".
{% endhint %}

Text nodes are also distributed in the default slot.

Use explicit slot names instead of relying on the default slot. The solution owner can remove the default slot from a view composition with the [CompositionEditor](https://github.com/Starcounter/Blending#blending-app-suite). As a result, all elements from the view that don't have a `slot` attribute are not rendered.

It's not necessary to declare the default slot in the `declarative-shadow-dom` part of your view. The Starcounter's HTML merger automatically adds a default slot at the bottom of the composition of your view for fallback reasons.

## Guideline 4: Create the Layout in `declarative-shadow-dom`

As outlined in guideline 1, the layout of the HTML view should be included within the `<template is="declarative-shadow-dom">`.

There is one exception to this. Both slot attributes and `declarative-shadow-dom` can be omitted if the view only contains non-visual elements or if all elements should be bulked together in the default slot. The latter is rarely the case but may be useful for example for prototyping.

The [`<slot>` element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/slot) is used to distribute the content in the Shadow DOM: `<slot name="appname/elementname"></slot>`.

Consider the following HTML view definition:

```markup
<link rel="import" href="/sys/palindrom-redirect/palindrom-redirect.html" />

<template>
    <template is="dom-bind">
        <style>
            .people-field {
                display: flex;
                flex-direction: row;
                align-items: baseline;
                margin-bottom: 5px;
            }
            .people-field__label {
                flex: 0 0 75px;
                margin-right: 20px;
            }
            .people-field__control {
                flex: 1 1 75px;
            }
        </style>
        <div class="people-field">
            <div class="people-field__label">
                <label slot="people/first-name-label" class="control-label">First name:</label>
            </div>
            <div class="people-field__control">
                <input slot="people/first-name-control" type="text" value="{{model.FirstName$::change}}" placeholder="First name" class="form-control" />
            </div>
         </div>
    </template>
</template>
```

To add `declarative-shadow-dom` to this HTML view, something like this can be done:

```markup
<link rel="import" href="/sys/palindrom-redirect/palindrom-redirect.html" />

<template>
    <template is="dom-bind">
        <label slot="people/first-name-label" class="control-label">First name:</label>
        <input slot="people/first-name-control" type="text" value="{{model.FirstName$::change}}" placeholder="First name" class="form-control" />
    </template>
    <template is="declarative-shadow-dom">
        <style>
            .people-field {
                display: flex;
                flex-direction: row;
                align-items: baseline;
                margin-bottom: 5px;
            }
            .people-field__label {
                flex: 0 0 75px;
                margin-right: 20px;
            }
            .people-field__control {
                flex: 1 1 75px;
            }
        </style>

        <div class="people-field">
            <div class="people-field__label">
                <slot name="People/first-name-label"></slot>
            </div>
            <div class="people-field__control">
                <slot name="People/first-name-control"></slot>
            </div>
         </div>
    </template>
</template>
```

Here, the elements are distributed in the way that the view will look when no blending is applied or when the app is running in standalone mode.

## Guideline 5: Apply Styling to Avoid Conflicts and Allow Blending

Regarding styling, there are two ways to make the application easier to visually integrate with other apps:

1. Prefix all class names with the name of the app, as outlined in [Avoiding CSS Conflicts](https://docs.starcounter.io/guides/blending/avoiding-css-conflicts/).
2. Keep styling that will affect the layout inside the `declarative-shadow-dom`.
3. To avoid writing the same Shadow DOM CSS on different pages, it can be imported with the CSS `import` rule. The syntax for this is `<style>@import url("/yourapp/css/style.css");</style>`. In the example above it would be done this way:

```markup
<link rel="import" href="/sys/palindrom-redirect/palindrom-redirect.html" />

<template>
    <template is="dom-bind">
        <label slot="People/first-name-label" class="control-label">First name:</label>
        <input slot="People/first-name-control" type="text" value="{{model.FirstName$::change}}" placeholder="First name" class="form-control" />
    </template>
    <template is="declarative-shadow-dom">
        <style>
        @import url("/people/css/style.css");
        </style>

        <div class="people-field">
            <div class="people-field__label">
                <slot name="People/first-name-label"></slot>
            </div>
            <div class="people-field__control">
                <slot name="People/first-name-control"></slot>
            </div>
         </div>
    </template>
</template>
```

## Additional Resources

To find more information about creating HTML View definitions, take a look at [the article linked above](https://starcounter.io/layout-compositions-html-partials/) and the [People app](https://github.com/StarcounterSamples/People/tree/develop/src/People/wwwroot/People/viewmodels) which fully adheres to these guidelines.


# Avoiding CSS conflicts

## Introduction

When your system is composed of HTML responses from multiple apps, there's always a risk of naming conflicts and cascading side effects in your CSS. A solution to this is to use a convention that isolates styles coming from different apps.

We recommend BEM, a well-known convention that solves this problem.

## What is BEM?

In the BEM naming convention, you **only use classes** in your stylesheets.  There are three kinds of classes at your disposal: blocks, elements, and modifiers.

A **block** is a basic class that represents a logical area of your app's UI (*a menu, login form, a search form*).

An **element** is a part of a block that performs a particular function (*a link in the menu, a password input in the login form, a search icon*).

A **modifier** is a variation of a block or of an element (*an expanded menu, an active menu item, a password input with an invalid value, a disabled search button*).

Possible combinations of blocks, elements and modifiers are the following:

```css
.block {}
.block__element {} /*The element part starts after __*/
.block--modifier {} /*The modifier part starts after --*/
.block__element--modifier {} /*Both blocks and elements can have modifiers*/
```

When applied to an HTML structure, the above CSS class names are used in the following way:

```markup
<div class="block">
  <div class="block__element"> ... </div>
  <div class="block__element--modifier"> ... </div>
</div>

<div class="block--modifier">
  <div class="block__element"> ... </div>
  <div class="block__element--modifier"> ... </div>
</div>
```

Note here that **everything at the root level must be a block**. A block can have multiple element and modifier sections and every element and modifier has to belong to a block.

## Example

Consider the [SignIn app](https://github.com/starcounterapps/signin):

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe50363f0-8680-11e7-9944-1f85270462c6%2FBEM-example.PNG?generation=1503327410904955\&alt=media)

Here, the block is marked in red and the element sections in blue.

From this, these BEM classes can be derived:

```css
.signin-form {}
.signin-form--expanded {}
.signin-form__text-input {}
.signin-form__labeled-checkbox {}
.signin-form__labeled-checkbox--checked {}
```

Check out the source code of [People](https://github.com/StarcounterApps/People) or [KitchenSink](https://github.com/StarcounterApps/KitchenSink). These sample apps show how to apply BEM in practice.

## BEM in Starcounter apps

We recommend the following rules when using BEM selectors in Starcounter apps.

* **Only use BEM class selectors in your stylesheets**. Do not use element selectors, id selectors, or inline styles for the purpose of styling.
* **Give meaningful names** to the block, element and modifier sections. For example, `.chatter-avatar` is much more descriptive than `.chatter-img`.
* **Use resusable names** for the block, element, and modifier sections. As seen in the example above, `.signin-form__text-input` is preferred over `.signin-form__firstname-input` since `text-input` is more resusable than `firstname-input`.
* **Prefix block sections with the app name** to isolate your classes from other apps. For example, the class for a menu block in the "Chatter" app should be `.chatter-menu`.
* **Use lowercase classes**. `.Chatter-Menu` is wrong, `.chatter-menu` is right.
* **Separate words with a hyphen** when there are multiple words in a block, element or modifier section. For example: `.chatter-chat-message__message-text`.
* **Block and element must be in the same HTML template**. Otherwise, implicit couplings are created between templates which might break when the partial mapping changes. If you want to define `.chatter-menu` in a parent partial and the menu items in a nested partials, these menu items will become new blocks (`.chatter-menu-item`, not `.chatter-menu__item`).
* **Modifier classes should extend base classes**.

  When you set a `.chatter-menu__item--active` class on an element, it should not be needed to add the `.chatter-menu__item` base class.

  In your stylesheet, the definition for the base class should include all the modifiers, like this:

  ```css
  .chatter-menu__item,
  .chatter-menu__item--active {
    font-size: 11px;
  }

  .chatter-menu__item--active {
    font-weight: bold;
  }
  ```
* **Never nest blocks inside blocks and elements inside elements.**

  If there’s need for more nesting, it means there’s too much complexity and the elements should be stripped down into smaller blocks.
* **Mixing BEM with Bootstrap**

  Starcounter sample apps use the CSS framework Bootstrap to create a unified look and feel.

  Even though Bootstrap does not follow BEM, there are no issues with mixing Bootstrap and BEM in a single project because there are no collisions. In fact, by just looking at the class, you can immediately tell if that class is shared with other apps (Bootstrap) or if it's private to this particular app (BEM).

  It is **not** fine to override Bootstrap classes in your app's stylesheet. The only proper way to extend style is to with a BEM selector, for example:

  ```markup
  <ul class="chatter-autocomplete">
    <template is="dom-repeat" items="{{model.FoundAttachment}}">
        <li class="chatter-autocomplete__item">
            <button type="button" class="btn btn-sm btn-link chatter-autocomplete__choose" onmousedown="++this.value;" value="{{item.ChooseTrigger$::click}}">{{item.NameAndType}}</button>
        </li>
    </template>
  </ul>
  ```

  For reference, the available Bootstrap classes can be found in [bootstrap.css](https://github.com/twbs/bootstrap/blob/v3.3.7/dist/css/bootstrap.css).

## Further reading

* [BEM: Key concepts (bem.info)](https://en.bem.info/method/key-concepts/)
* [Naming convention (bem.info)](https://en.bem.info/method/naming-convention/)
* [BEM-like Naming (cssguidelin.es)](http://cssguidelin.es/#bem-like-naming)
* [MindBEMding – getting your head ’round BEM syntax (csswizardry.com)](http://csswizardry.com/2013/01/mindbemding-getting-your-head-round-bem-syntax/)
* [An Introduction to the BEM Methodology (tutsplus.com)](http://webdesign.tutsplus.com/articles/an-introduction-to-the-bem-methodology--cms-19403)


# Debugging

## Introduction <a href="#introduction" id="introduction"></a>

Since views in Starcounter are built on a thin-client architecture and use a different communication protocol - Palindrom instead of REST - they're debugged in a slightly different way. This page describes how to debug views in Starcounter.

## Starcounter DevTools <a href="#starcounter-devtools" id="starcounter-devtools"></a>

The primary tool for debugging Starcounter apps is [Starcounter DevTools](https://github.com/Starcounter/DevTools). Starcounter DevTools is a [WebExtension](https://browserext.github.io/browserext/) that lets you inspect view-models, patches sent over Palindrom, and imported packages.

You can download Starcounter DevTools for [Chrome](https://chrome.google.com/webstore/detail/starcounter-devtools/mpchkilmmalfopikamgellgdgoidhmnh) and [Firefox](https://addons.mozilla.org/en-US/firefox/addon/starcounter-devtools/). For other browsers, you can add this code as a bookmarklet:

```javascript
javascript:(function(){var script = document.createElement('script');script.src = 'https://rawgit.com/Starcounter/DevTools/master/build/webextension/injected_script.js';document.body.appendChild(script);script.onload=()=>window.dispatchEvent(new CustomEvent('sc-debug-show-overlay'))})()
```

## Investigating patches <a href="#investigating-patches" id="investigating-patches"></a>

Starcounter apps communicate over Palindrom - a protocol that uses WebSocket with JSON-Patch. When debugging, it's useful to see the patches that are sent and received. This can be done with both Starcounter DevTools and Chrome DevTools.

With Starcounter DevTools, use the "Patches" tab. This tab shows all requests and responses, both HTTP and WebSocket. To filter out the type of requests and responses, use the drop-down in the top left corner. There, you can choose to only see, for example, WebSocket patches or GET requests. The drop-down next to the type drop-down lets you only see requests or responses. 

![Patches view in Starcounter DevTools](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L329JLtK2uIFjG_-bZZ%2F-L32ACaNKirdy8kbng7B%2FCapture.PNG?alt=media\&token=058c80ca-1e97-496c-8294-8c4cd765af1f)

To investigate patches in Chrome DevTools, go to Network, select WS to only see WebSocket patches, and click the top WebSocket request. If you opened the DevTools after Palindrom established the WebSocket connection, you have to refresh the page to see the connection in DevTools. You can then see all the patches and their content. 

![Investigating patches in Chrome DevTools](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32CRaNu7V67gZ3blyQ%2F-L32DqR8xzUpIy3xZqJI%2FChromeDevTools.PNG?alt=media\&token=543c0aa8-fa51-4551-9c39-61a1ccfce216)

## Investigating view-models <a href="#investigating-view-models" id="investigating-view-models"></a>

To investigate the view-models of a page, use the "View-model" tab of Starcounter DevTools. It lets you view the view-models on the current page, either as an expandable tree or as a plain text JSON tree 

![](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32CRaNu7V67gZ3blyQ%2F-L32EYXOiIIAXH1JklcD%2FDevToolsViewModels.PNG?alt=media\&token=3f01846f-8449-4ee8-82f5-978538c56954)

If a property in the tree is not populated as expected - check whether the name in the HTML matches the name of the JSON property name.

## Investigate versions of imported dependencies <a href="#investigate-versions-of-imported-dependencies" id="investigate-versions-of-imported-dependencies"></a>

There are two ways to check the version of an imported client-side dependency. The first way is to check the header of the imported file with Chrome DevTools. To do this, open DevTools, go to the Network tab, click "Preview". At the top of the file, the author of the dependency should have written the version. 

![Checking version in Chrome DevTools](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32KuNEAk96khEwhpBb%2F-L32LLSp1ikSii95GYod%2FVersionChrome.PNG?alt=media\&token=3daa27e9-442e-49d6-9514-c0744fbc86af)

The second way is to use the Imports tab of Starcounter DevTools. There you can click "Get Bower versions" to retrieve the Bower version of all the dependencies. 

![View versions in Starcounter DevTools](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32KuNEAk96khEwhpBb%2F-L32LotsQv27Se-DiSmZ%2FStarcounterDevToolsVersion.PNG?alt=media\&token=5af29216-665f-473b-abb2-304308f4ef30)

To check where the file is imported from, set the [database configuration](/2.3.1/guides/database/database-configuration) option `X-File-Path Header` to `true`. By doing this, you can see whether the imported file has been overridden by another app. This option should be disabled in production. 

![Enabling X-File-Path](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32NN4YfOeCymWTTrQ1%2F-L32OKLYIUnmBE6boV3K%2FChangeXPath.gif?alt=media\&token=d074d853-3ed3-4e5f-a06a-d0e27bab8c74)

With `X-File-Path` enabled, you can use Chrome DevTools to find the exact path by locating the header `X-File-Path` in the Network tab. 

![Finding the X-File-Path](https://blobscdn.gitbook.com/v0/b/gitbook-28427.appspot.com/o/assets%2Fstarcounter%2F-L32NN4YfOeCymWTTrQ1%2F-L32PdFQgs-2W3i5Sxuc%2FXFilePath.PNG?alt=media\&token=8d85693f-c885-4d21-8165-0fb638a51ab9)


# Network

## Introduction

Starcounter communicates with the standard web protocols HTTP, WebSocket, TCP, and UDP. Starcounter is efficient at handling requests with standard high level web standards such as REST and JSON.

While REST is a standard option for web apps, Starcounter has also built-in support for creating stateful web apps using JSON-Patch.

Starcounter apps can make `Self` requests, which can be used for internal signalling or composing responses from smaller bits and pieces.


# HTTP

## Introduction

There are two ways of doing routing in Starcounter:

1. With the built-in `Handle` API that handles HTTP
2. With the routing API found in the Authorization library

In most cases, it's recommended to use the API in the [Authorization library](https://github.com/Starcounter/authorization#routing-middleware-and-context---concepts) because of its ease of use. Although, there might be some cases where the `Handle` API is needed.\
This page describes the built-in `Handle` API

## Requests

### Catching Incoming Requests

Incoming HTTP 1.0/1.1 requests are caught using the static `Handle` class.

```csharp
Handle.GET("/hello", () =>
{
    return "Hello World";
});
```

Handlers can be registered anywhere and at any time. Though, in most cases, they are registered in the `Main` method

### Catching Common HTTP Verbs

The basic HTTP methods `GET`, `POST`, `PUT`, `DELETE` and `PATCH` can be caught by methods with the same name in the `Handle` class.

```csharp
Handle.GET("/hello", () =>
{
    return "Hello World";
});

Handle.POST("/hello", () =>
{
    return 500;
});

Handle.DELETE("/hello", () =>
{
    return 500;
});
```

### Accepting Parameters in Requests

When matching incoming requests, some parts of the URI may contain dynamic data. This is handled by Starcounter by allowing you to define parameters in handlers. This is done by marking the dynamic part of the URI template with curly braces. The simplest use of the curly brace syntax is a single question mark `{?}`. This indicates that there is a fragment of dynamic data in the URI. The type of the data is determined by the code delegate that follows.

```csharp
Handle.GET("/hello/{?}", (string name) => 
{         
    return "Hello " + name;
});
```

In the above example, the delegate accepts `string name`. This means that the parameter will be parsed as a string, for example: `/hello/albert`, `/hello/anna`. To accept an integer parameter, simply change the lambda parameter type.

```csharp
Handle.GET("/squared?{?}", (int num) => 
{         
    return num + " squared equals " + num * num;
});
```

The accepted URIs would, for example, be `/squared?123` and `/squared?-4321`

To accept multiple dynamic fragments, add more curly braces. For each dynamic parameter there should be a parameter in the delegate. They are enumerated from left to right, so be careful to put the parameters in the right order.

```csharp
Handle.GET("/{?}/{?}", (string list, int item) => 
{         
    return "List is " + list + " and item is " + item;
});
```

The accepted URIs would be, for example: `/serialnumbers/4534123`, `/itemid/34321`

#### Database object as parameter in handler&#xD;

One can also expect a database object as a parameter to a handler:

```csharp
[Database]
public class Person
{
    public string Name { get; set; };
}
```

```csharp
Handle.GET("/People/{?}", (Person person) => person.Name);
```

Such handler can be called with database object ID as a parameter or using special `Self.GET` variant, which takes database class instance as a second parameter:

```csharp
var person = Db.SQL<Person>(...).FirstOrDefault();
Response response = Self.GET("/People/{?}", person);
```

The handler above can be also called with object ID as parameter:

```csharp
string objectId = DbHelper.GetObjectID(c);
Response response = Self.GET("/People/" + objectId);
```

### Catching Other Verbs

The `CUSTOM` verb in the `Handle` class makes it possible to register other HTTP methods or even catch all methods and URIs.

```csharp
Handle.CUSTOM("REPORT /hello/{?}", (string p1) =>
{
    return 500;
});

Handle.CUSTOM("{?} /hello/{?}", (string method, string p1) =>
{
    return p1;
});

Handle.CUSTOM("OPTIONS", "/hello/{?}", (string p1) =>
{
    return p1;
});

Handle.CUSTOM("{?}", (string methodAndUri) =>
{
    return "Caught: " + methodAndUri;
});
```

### The `Request` Object

A `Request` parameter can be declared together with the enumerated parameters. It encapsulates the entire request.

```csharp
Handle.GET("/hello", (Request request) =>
{
    return 500;
});

Handle.GET("/persons/{?}", (string name, Request request) =>
{
    return 500;
});
```

To access certain request HTTP headers, use `Headers[String]` accessor on a `Request` object (same as for the `Response` object):

```csharp
string mySuperHeader = req.Headers["MySuperHeader"];
string allRequestCookies = req.Headers["Set-Cookie"];
```

Request cookies are accessible from `Cookies` as a list of strings "name=value" (same as for Response object):

```csharp
List<String> allRequestCookies = req.Cookies;
```

To obtain client IP address, use `GetClientIpAddress()` on the `Request` object.

Use the `HandlerAppName` property to find out which application the request belongs to. This might be useful when working with request filters.

### Handler Options

When creating (using the `Handle` interface) and calling handlers (using the `Self` interface), one can supply last `HandlerOptions` parameter, which specifies certain options for handler calls or registration. Here are the notable handler options:

* `SkipRequestFilters`: used to declare a handler for which request filters will not be applied.
* &#x20;`SkipResponsetFilters`: used to declare a handler for which response filters will not be applied.
* `SkipHandlersPolicy`: If the database flag "EnforceURINamespaces" is set to True, all application handlers are required to start with application name. In this case `SkipHandlersPolicy` flag allows to register any URI handler.
* `SelfOnly`: registered handler is going to be accessible only inside codehost using `Self` interface. `SelfOnly` handlers are not registered in gateway, in comparison with normal handlers.

**Examples**:

Registering a handler that skips request filters:

```csharp
Handle.POST("/myhandler", (Request request) =>
{
    return 204;
}, new HandlerOptions() { SkipRequestFilters = true });
```

### Exception Propagation Within Applications

Internal requests are requests made to handlers within same user application (`sccode` instance) using `Node` and `X`. Internal requests and handlers can be nested and create a call hierarchy. Sometimes its useful to cast a specific exception deep down in the hierarchy and handle it on another level or let the system handle it (for example by automatically sending the response). This can be achieved using `ResponseException` exception object. The following example illustrates this concept:

```csharp
Handle.GET("/exc1", (Request req) =>
{
    Response resp = Self.GET("/exc2");
    return resp;
});

Handle.GET("/exc2", (Request req) =>
{
    try
    {
        Response resp = Self.GET("/exc3");
        return resp;
    }
    catch (ResponseException exc)
    {
        exc.ResponseObject.StatusDescription = "Modified!";
        exc.ResponseObject.Headers["MyHeader"] = "Super value!";
        exc.UserObject = "My user object!";
        throw exc;
    }
});

Handle.GET("/exc3", (Request req) =>
{
    Response resp = new Response()
    {
        StatusCode = 404,
        StatusDescription = "Not found!"
    };

    throw new ResponseException(resp);
});
```

Handler `/exc3` constructs and throws an instance of `ResponseException` exception. Handler `/exc2` catches the exception, modifies some data and re-throws the exception. Eventually, the `ResponseException` is caught by outer system handler and `ResponseObject` is automatically sent on the original `Request req`. Note that `ResponseException` mechanisms are working only within one user application (they are simple C# exceptions).

User can attach an arbitrary user object to `ResponseException` by either constructor or `UserObject` property.

### Unregistering HTTP Handlers

After an HTTP handler is created - it can be unregistered with the `Handle.UnregisterHttpHandler` method.

### Size Limit on Payloads

The default limit on payloads in requests is 1048576 bytes, exceeding this limit will prevent the request from going through and this warning will be printed in the [Administrator log](/2.3.1/guides/working-with-starcounter/administrator-web-ui#log):&#x20;

> Attempt to HTTP upload of more than 1048576 bytes. Closing socket connection

The limit can be increased to a maximum of 2048576 bytes by changing the value of  `MaximumReceiveContentLength` in `%userprofile%\Documents\Starcounter\Personal\scnetworkgateway.xml`.

{% hint style="info" %}
When sending large files, we recommend to use WebSocket instead of HTTP. That pattern is demonstrated in the `FileUploadPage` ([code-behind](https://github.com/StarcounterApps/KitchenSink/blob/master/src/KitchenSink/FileUploadPage.json.cs), [HTML](https://github.com/StarcounterApps/KitchenSink/blob/master/src/KitchenSink/wwwroot/KitchenSink/FileUploadPage.html)) in the sample app [KitchenSink](https://github.com/StarcounterApps/KitchenSink).
{% endhint %}

## Responses

When responding to a request from a handler such as `Handle.GET`, a `Response` object should be returned.

The `Response` class has many implicit cast operators to make this convenient.

* `string` (mime type will be text/plain or text/html depending on the request `Accept` header)
* `byte[]` (mime type will be the first one in the `Accept` header)
* `Json` object (mime type will be `application/json`)
* `int`, `uint`, `decimal`, `bool`, `double`, `long`, `ulong`, `DateTime` returns a Javascript literal (JSON)
* `null` (no content)

### Returning Different Types

#### Response Object

When creating a `Response` object, you have the choice of setting the body to a `byte[]`, a `string`.

```csharp
Handle.GET("/hello", () =>
{
    new Response()
    {
        ContentType = "text/plain",
        Body = "Hello World"
    };
});
```

#### JSON Object

When returning an instance of the `Json` class, the mime type will `application/json` and the body will contain a JSON string.

```csharp
Handle.GET("/hello", () =>
{
    return new PersonData()
    {
        FirstName = "Joachim",
        LastName = "Wester"
    };
});
```

#### String

When returning a string, the returned mime type depends on the `Accept` header of the request. If the request prioritizes `text/html` or `application\json`, the HTTP response will use this type accordingly. If no Accept header was provided, the mime-type `text/plain` will be used.

```csharp
Handle.GET("/hello", () => "Hello World" );
```

#### Status Code and Status Description

Create a `Response` object:

```csharp
GET("/hello", () =>
{
    return new Response()
    {
        StatusCode = 404,
        StatusDescription = "Not Found"
    };
});
```

If an integer is returned from a delegate that will result in automatic response with a `StatusCode` equal to the integer and default status description.

#### Null

When null is returned from the handler, it's equal to returning the 404 `Not found` status code.

#### Streamed Body

When creating the `Response` object, the body can be set to a `byte[]`, `string`, or `Stream`.\
As Starcounter schedules threads in a optimized way, it is recommended to allow Starcounter to handle streaming. This is done by assigning the `stream` to the `Response` object and then returning the response, relying on Starcounter to read the stream and send the network traffic on its own accord. Streamed object should allow getting length of the stream in bytes (`Length` property).

```csharp
Handle.GET("/movie", () =>
{
   FileStream stream = File.Open("bigfile.mpg", FileMode.Open, FileAccess.Read, FileShare.Read));
   var response = new Response()
   {
        StreamedBody = stream,
        ContentType = "application/octet-stream"
   };
   return response;
});
```

Note that the `stream` object is automatically closed when the stream data is sent completely or if the connection is dropped.

### Serving Static Resources

To resolve a static resource, there is a method `Handle.ResolveStaticResource` which takes a resource URI and incoming request and returns a response representing this resource. Response however can be a 404, so to return a "nice" 404 page user has to do the following code, for example:

```csharp
Response resp = Handle.ResolveStaticResource(req.Uri, req);

if (404 == resp.StatusCode)
{
    resp = Self.GET("/404.html");
}
return resp;
```

When the codehost starts, Starcounter adds a static resource resolver on the default user port (GET handler on URI "/{?}").

### Delayed or Explicitly Handled Responses

Sometimes, the `Response` object cannot be returned immediately in the handler. One reason could be the access of third party resources or doing long-running jobs. By returning `HandlerStatus.Handled` in the handler, the user indicates that the response will be returned later or that it already has been returned another way.

For example:

```csharp
Handle.GET("/postponed", (Request req) =>
{
    Http.POST("/posttest", "Here I do a post!", null, null, (Response resp, Object userObject) =>
    {
        // Modifying the response object by injecting some data.
        resp.Headers["MySuperHeader"] = "Here is my header value!";
        resp.Headers["Set-Cookie"] = "MySuperCookie=CookieValue";
        req.Response = resp;
    }); // "resp" object will be automatically sent when delegate exits.
    return HandlerStatus.Handled;
});
```

Please refer to the [External HTTP calls and Node usage](/2.3.1/guides/network/external-http-calls) article for more information.

### Summary of Adjustable Response Fields

* `StatusCode`: 404, 501, etc
* `StatusDescription`: *"Not Found"*, *"Service Unavailable"*, etc
* `ContentType`: *"text/html"*, *"application/json"*, etc
* `ContentEncoding`: *"gzip"*, etc
* `Cookies`: *a list of entries like "MyCookie1=123", "MyCookie2=456"*, etc
* `Body`: *"Here is response body!"*, etc
* `BodyBytes`: *Byte\[] bodyBytes = { 1, 2, 3, 4, 5};*, etc
* `ConnFlags`: used to manipulate the connection with client.

The following `Response.ConnectionFlags` values are available:

* `Response.ConnectionFlags.DisconnectImmediately`: immediately disconnects the associated connection with endpoint without sending any data first.
* `Response.ConnectionFlags.DisconnectAfterSend`: first sends given message to endpoint and then closes the corresponding connection.

Example:

```csharp
Handle.GET(8080, "/shutdown", (Request req) =>
{
    return new Response()
    {
        Body = "Closing connection with you!",
        ConnFlags = Response.ConnectionFlags.DisconnectAfterSend
    };
});
```

Setting arbitrary HTTP headers on a `Response` object is straightforward using the `Headers` property:

```csharp
Handle.GET("/response", () =>
{
    var response = new Response()
    {
        StatusCode = 404,
        StatusDescription = "Not Found",
        ContentType = "text/html",
        ContentEncoding = "gzip",
        Cookies = { "MyCookie1=123", "MyCookie2=456" },
        Body = "response1"
    };

    response.Headers["MyHeader"] = "MyHeaderData";
    response.Headers["Location"] = "/newlocation";

    return response;
});
```

Remarks:

* `StatusCode` default value is 200
* `StatusDescription` default value is *"OK"*
* `Server` default value is `Server: Starcounter/#starcounter_version (Windows)`, for example: `Server: Starcounter/2.3.1.7779 (Windows)`.
* To access certain HTTP headers, use the `Headers` accessor on the `Response` or `Request` object:

```csharp
Response response;
...
if ("SC" == response.Headers["Server"])
{
  return response.Headers["Accept-Ranges"];  
}
```

Examples:

```csharp
Handle.GET("/hello", () =>
{
  return new Response()
  {
    StatusCode = 404,
    StatusDescription = "Not Found",
    ContentType = "text/html",
    ContentEncoding = "gzip",
    Cookies = { "MyCookie1=123", "MyCookie2=456" },
    Body = "response1"
  };
});
```

{% hint style="info" %}
Encode complex HTTP header values, such as those containing line separators, using base64 or similar encoding to reduce the chance of headers that are parsed the wrong way.
{% endhint %}

### Setting Properties on Outgoing Response

Sometimes, when deep in the call hierarchy of `Self.GET`, it's necessary to be able to set properties not on the current response, but directly on corresponding, not yet available, outgoing responses, like headers, status codes, messages, and cookies. To achieve that, the following static methods are available from the class `Handle`:

```csharp
void AddOutgoingCookie(String cookieName, String cookieValue);
void AddOutgoingHeader(String headerName, String headerValue);
void SetOutgoingStatusDescription(String statusDescription);
void SetOutgoingStatusCode(UInt16 statusCode);
```


# Internal Self Calls

## Introduction

Starcounter provides an efficient way for REST communication within the codehost instance. Simply put, `Self` is used to call handlers that are registered using the `Handle` class inside the codehost. To communicate between different codehosts, `Http` should be used. `Self` communication does not use either networking or shared memory, so it is very efficient. It is represented by the `Self` class, which is similar to the `Http` interface. \
For example, the same HTTP methods are supported, as in `Http`. However, in comparison, `Self` calls are always synchronous, so asynchronous mode is not presented in it. Like for `Http`, the `Response` object is returned as a result of `Self` call. To conclude, `Self` is used ubiquitously in Starcounter as it is the core REST communication mechanism.

## Usage

Here are some examples of `Self` calls:

```csharp
Response resp = Self.GET("/MyHandler");
```

Templated `Self` can be used to specify what object type is expected in `Body` of the `Response` and gets it as a return value, for example:

```csharp
Json json = Self.GET<Json>("/MyApp/MyJsonObject/13235");
```

Here, an object of type `Json` is expected to be in the `Body`.

A specific JSON type can also be used:

```csharp
Master master = Self.GET<Master>("/emails");
```

Here is an example of expecting and obtaining the string `Body`:

```csharp
String myText = Self.GET<String>("/MyApp/MyTextDocument/54664");
```

or expecting a binary body:

```csharp
Byte[] myBinaryData = Self.GET<Byte[]>("/EncodedDocument/34563");
```

Note that if the actual response `Body` object returned in handler is of different type than expected - the conversion exception will be thrown.

## Getting Current Level in the Call Hierarchy

The hierarchy of `Self` calls can be quite deep and sometimes its needed to get the current call level. To achieve that there is a special thread static variable `Handle.CallLevel`. Every `Self` call the variable is incremented and then restored to current value on the way back.


# Middleware

## Introduction

Middleware is code that affects the *request pipeline*. It enables applications to customize how the server handles requests. This customization comes in three different shapes:

1. request filters
2. response filters
3. response and request filters wrapped up in *middleware classes*.

These can help with a range of issues, such as security, logging, wrapping, request modification, and more.

An example of this is the [Launcher](https://github.com/starcounterapps/launcher) which uses request filters to wrap responses from other applications into its own response.

All middleware is registered with the `Application.Current.Use` API which has three overloads corresponding to the different types of middleware listed above. The registration is usually done at the entry point of the application and can look something like this:

```csharp
static void Main()
{
  Application.Current.Use((Request request) => {
    // Request filter goes here
  });

  Application.Current.Use((Request request, Response response) => {
    // Response filter goes here
  });

  Application.Current.Use(new SomeMiddleware());
}
```

Middleware does not impact internal `Self.GET` calls.

## Request Filters

When allowing external HTTP requests, it might be useful to pre-process or filter out certain requests before the designated handler is called. Request filters make it possible to do exactly that. They are lists of user-supplied delegates, or filters, that are executed on external requests before the actual handlers are called. These filters are executed one by one until one of the filters returns a non-null `Response`. If a `Response` was returned from the request filter, then this response is returned to the client without calling the handler. If none of the filters returned a `Response` object, then the request will be passed on and dealt with by the handler.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe58209d0-8680-11e7-9944-1f85270462c6%2FMiddleware-example.PNG?generation=1503327411740456\&alt=media)

An example of this can be an basic spam filter:

```csharp
Application.Current.Use((Request request) =>
{
    if (request.Uri.Contains("spam"))
    {
        return new Response()
        {
            StatusCode = 406,
            StatusDescription = "Well, thanks, but no thanks!"
        };
    }
    return null;
});
```

When there is an incoming request, this request filter checks if the URI contains the string "spam", and returns a `Response` object if that's the case. This means that the request will be blocked without reaching the handler. Otherwise, it returns `null` and allows the request to move on to the next request filter or go to the handler if there was only one request filter.

### Skip Request Filters

To let requests to a certain handler bypass all request filters, use the class `HandlerOptions` and set `SkipRequestFilters` to `true`. Like this:

```csharp
Handle.GET("/my-url", () => new Json(), new HandlerOptions() { SkipRequestFilters = true });Response Filters
```

## Response Filters

Response filters do the opposite of request filters; they make alterations to outgoing responses. They work similarly to request filters by being executed one by one until one returns a non-null response. The main difference is that response filters are called after the handler has been called while request filters are called before. Response filters can either create entirely new responses and return those, or modify the response coming from the handler.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe58230e0-8680-11e7-9944-1f85270462c6%2Fmiddleware-response.PNG?generation=1503327413340430\&alt=media)

For example, response filters makes it possible to add a certain HTTP header to responses for requests with a `/special` URI prefix after the request has been dealt with by the handler:

```csharp
Application.Current.Use((Request request, Response response) =>
{
    if (request.Uri.StartsWith("/special"))
    {
        response.Headers["MyHeaderName"] = "MyHeaderValue";
        return response;
    }
    return null;
});
```

In this case, a new header would be added to the response if the URI of the incoming request started with `/special`. It would then return the response and no other response filters would be called. If the request URI did not start with `/special`, then the next response filter would be called or the response would be returned if there were no more response filters to call. Take a look at this response filter by comparison:

```csharp
Application.Current.Use((Request request, Response response) =>
{
    if (request.Uri.StartsWith("/special"))
    {
        response.Headers["MyHeaderName"] = "MyHeaderValue";
    }
    return response;
});
```

In this case, the next response filter is never called since a response is always returned. That is important to keep in mind when building response filters.

In the examples above, the response filter checks for information in the request. It's also possible to check for information in the response, such as in this example:

```csharp
Application.Current.Use((Request request, Response response) =>
{
    if (response.StatusCode == 404)
    {
        return new Response()
        {
            StatusCode = 404,
            StatusDescription = "Not Found",
            Body = Self.GET("/myapp/404.html").Body
        };
    }
    return null;
});
```

Here, the response filter makes it possible to return a descriptive `404` page by checking the outgoing responses for the `404` status code and return a response containing the "not found" HTML page.

### Skip response filters

For a handler to bypass all response filters, use the class `HandlerOptions` and set `SkipResponseFilters` to `true`. Like this:

```csharp
Handle.GET("/my-url", () => new Json(), new HandlerOptions() { SkipResponseFilters = true });
```

## Response and Request Filter Interaction

When using both request and response filters, response filters will intercept responses coming from request filters. Consider the following example:

```csharp
static void Main()
{
    // Request filter
    Application.Current.Use((Request request) =>
    {
        return new Response()
        {
            StatusCode = 404,
            StatusDescription = "Not Found",
            Body = "THIS IS FROM THE REQUEST FILTER"
        };
    });

    // Response filter
    Application.Current.Use((Request request, Response response) =>
    {
        if (response.StatusCode == 404)
        {
            return new Response()
            {
                StatusCode = 404,
                StatusDescription = "Not Found",
                Body = "THIS IS FROM THE RESPONSE FILTER"
            };
        }
        return null;
    });

    Handle.GET("/Test", () =>
    {
        return "This will never be seen";
    });
}
```

When sending a request to the `/Test` handler, the request filter will intercept it and create a `Response` object. This response is then intercepted by the response filter that creates another `Response` object which will be the one received by the client. When this code runs, "THIS IS FROM THE RESPONSE FILTER" displays.

## Middleware Interaction with Other Applications

Both request and response filters catch requests to handlers in other applications that are running simultaneously.

Consider the following request filter:

```csharp
Application.Current.Use((Request request) =>
{
    return new Response()
    {
        StatusCode = 404,
        StatusDescription = "Not Found",
        Body = "THIS IS FROM THE REQUEST FILTER"
    };
});
```

By having this request filter in one application, all requests, to all applications, will hit the `404` response.

Adding this `if` statement fixes it:

```csharp
Application.Current.Use((Request request) =>
{
    if (request.Uri.StartsWith("/MyApp"))
    {
        return new Response()
        {
            StatusCode = 404,
            StatusDescription = "Not Found",
            Body = "THIS IS FROM THE REQUEST FILTER"
        };
    }
    return null;
});
```

## Middleware Classes

`Application.Current.Use` also allows for exposing custom middleware classes. These middleware classes implement the `IMiddleware` interface.

Here's an application using a middleware class:

```csharp
class Blocker : IMiddleware
{
    void IMiddleware.Register(Application application)
    {
        application.Use((Request request) =>
        {
            return new Response()
            {
                StatusCode = 500,
                StatusDescription = "Blocker doesn't allow ANYTHING to get through!"
            };
        });
    }
}

class Program 
{
    static void Main()
    {
        Application.Current.Use(new Blocker());

        Handle.GET("/blocked", () =>
        {
            return "No one will call me :( ";
        });
    }
}
```

Here, we use the `IMiddleware` interface to create a custom middleware class. The middleware created is a request filter that returns an unhelpful response no matter what the incoming request is.

Middleware classes makes it possible to hide the implementation of middleware.

These classes do not have to contain request or response filters. Although, that is the common way of using middleware classes.

### HtmlFromJsonProvider

`HtmlFromJsonProvider` is a custom middleware class provided by Starcounter. It acts as a response filter by intercepting outgoing responses containing JSON objects and instead responding with the corresponding HTML. For example, look at the following application:

```markup
<template>
    <template is="dom-bind">
        <h1>{{model.FirstName}}</h1>
        <h3>{{model.LastName}}</h3>
    </template>
</template>
```

```csharp
{
  "Html": "/person.html",
  "FirstName": "John",
  "LastName": "Doe"
}
```

```csharp
void Main() 
{
  Application.Current.Use(new HtmlFromJsonProvider());

  Handle.GET("/person", () =>
  {
    return new Person();
  });
}
```

When there is a call to`/person`, this is what will happen:

1. The handler returns The JSON object `Person`
2. The `HtmlFromJsonProvider` intercepts the object
3. It looks in the JSON file for the `Html` property and finds `/person.html`
4. It returns the HTML found at that path

If the HTML at the path would be a complete HTML document, this would be enough. Though, because the HTML provided in this example is a HTML template, it's necessary to add another layer of middleware to convert the template to an HTML document that the browser can render. That's what `PartialToStandaloneHtmlProvider` does.

#### ScErrInvalidOperation

&#x20;If you have a JSON file without an \`Html\` property, `HtmlFromJsonProvider` will throw this exception:

```
System.InvalidOperationException: ScErrInvalidOperation (SCERR1025): Operation invalid for the object's current state. Json instance MyJson missing 'Html' property.
```

\
If the JSON has a corresponding HTML file, add an \`Html\` property with the path to the HTML file. That will fix it.

&#x20;If you don't have an `Html` property and don't intend to return HTML, but to return the JSON instead, set `IgnoreJsonWithoutHtml` to `true`:

```csharp
Application.Current.Use(new HtmlFromJsonProvider() { IgnoreJsonWithoutHtml = true });
```

### PartialToStandaloneHtmlProvider

This middleware class checks if the HTML is a full document, or essentially if it starts with a `<!DOCTYPE html>`. If it's not a full HTML document, it wraps the existing HTML inside the body of an HTML document that contains the following:

1. A `palindrom-client` element to create a WebSocket connection
2. Import links to the Starcounter custom elements `starcounter-include` and `starcounter-debug-aid`
3. Import links to the outside libraries Polymer and Bootstrap
4. The session URL which makes it possible for PuppetJs to request the relevant JSON in a future request

It's possible to override this default HTML by passing a string containing HTML as a parameter. Here's an example of that:

```csharp
var html = @"<!DOCTYPE html>
<html>
<head>
    <meta charset=""utf-8"">
    <title>{0}</title>
    <script src=""/sys/webcomponentsjs/webcomponents.min.js""></script>
    <script src=""/sys/document-register-element/build/document-register-element.js""></script>
    <link rel=""import"" href=""/sys/polymer/polymer.html"">
    <link rel=""import"" href=""/sys/starcounter.html"">
    <link rel=""import"" href=""/sys/starcounter-include/starcounter-include.html"">
    <link rel=""import"" href=""/sys/starcounter-debug-aid/src/starcounter-debug-aid.html"">
    <link rel=""import"" href=""/sys/bootstrap.html"">
    <style>
        body {{
            margin: 20px;
        }}
    </style>
</head>
<body>
    <template is=""dom-bind"" id=""puppet-root"">
        <template is=""imported-template"" content$=""{{{{model.Html}}}}"" model=""{{{{model}}}}""></template>
    </template>
    <palindrom-client ref=""puppet-root"" remote-url=""{1}""></palindrom-client>
    <starcounter-debug-aid></starcounter-debug-aid>
</body>
</html>";
Application.Current.Use(new PartialToStandaloneHtmlProvider(html))
```

Since `PartialToStandaloneHtmlProvider` wraps the actual response from the handler, it will have the HTTP status code of the response.

When building Starcounter applications the recommended way, use the two middleware classes mentioned above to send complete HTML documents to the client even if the handler returns Typed JSON classes.

This is how it should look:

```csharp
static void Main()
{
  Application.Current.Use(new HtmlFromJsonProvider());
  Application.Current.Use(new PartialToStandaloneHtmlProvider());

  // Rest of the application
}
```


# Anonymous or Substitute Handlers

## Introduction

Sometimes, it is required to get a partial without declaring a handler. This is when anonymous handlers became handy.

```csharp
model.Partial = Self.GET("/custom-url", () =>
{
    return new Page()
    {
        Html = "/custom-page.html"
    };
});
```

The `model.Partial` will contain the returned `new Page()` and all the responses from all the apps mapped to the `"/custom-url"`. The anonymous handler can be called to any url.

## Things to Know About Anonymous Handlers

* The anonymous handlers should never be used for direct calls to handlers defined in other apps. Follow the practice that the apps should not be aware of each other.
* An app can only have one response per URL, that is why anonymous handlers can't be combined with declared ones.
* The `Html` property is not required, it may be blank if the calling app does not have any response.
* The anonymous handler response should be of type `Starcounter.Page` otherwise it won't be possible to merge with other responses.

## When to Use Anonymous Handlers

### Shared UI Sections

The most common case is the [Launcher](https://github.com/starcounterapps/launcher). The Launcher defines different UI sections and populates them with anonymous `Self.GET` requests.

```csharp
//UriMapping.MappingUriPrefix - a constant, equals to "/sc/mapping"
model.Menu = Self.GET(UriMapping.MappingUriPrefix + "/menu", () => new Page());
```

The `model.Menu` contains merged responses from all handlers mapped to `/sc/mapping/menu`. The mapping would look like this:

```csharp
UriMapping.Map("/Products/menu", UriMapping.MappingUriPrefix + "/menu");
```

Read more about mapping here: [Mixing apps](broken://pages/-KtAuTaya83VsAsPECBV).

### UI View Composition with Layout Editor

Composition technology requires some extra HTML elements and JSON wrapping which are injected per partial [using middleware](/2.3.1/guides/network/middleware). This means that partials which are created without a request won't be available for view composition.

```csharp
// The IndexPage is available for view composition.
// It is retrieved with a request.
Handle.GET("/my-app", () => new IndexPage());
```

```csharp
Handle.GET("/my-app", () => new MasterPage()
{
    // The IndexPage is not available for view composition.
    // It is retrieved without a request.
    CurrentPage = new IndexPage()
});
```

```csharp
// The MasterPage and the IndexPage are both retrieved
// with a request and available for view composition.
Handle.GET("/my-app", () => MasterPage()
{
    CurrentPage = Self.GET("/my-app/partials/index")
});

Handle.GET("/my-app/partials/index", () => new IndexPage());
```

An anonymous handler can be used instead of the custom partial handler definition.

```csharp
// The IndexPage is still available for view composition,
// It is retrieved with an anonymous request.
Handle.GET("/my-app", () => MasterPage()
{
    CurrentPage = Self.GET("/my-app/partials", () => new IndexPage())
});
```


# URL Aliases and Redirects

## Introduction

With HTTP handlers, it is possible to express URL transformations such as aliases and redirects.

## Alias

An alias is a `GET` handle that returns the result of an `Self.GET` call.

For web apps, the URL displayed in the browser address bar does not get changed to the destination value.

Example:

```csharp
Handle.GET("/pmail", () =>
{
  return Self.GET("/pmail/inbox");
});
```

Starcounter gateway has support for aliases which is described in Network Gateway section.

## Redirect

An HTTP redirect is a server response that guides the client to access the resource using a different URL. It is achieved by an appropriate HTTP status code and value of the `Location` header.

For web apps, the URL displayed in the browser address bar gets changed to the destination value.

Example:

```csharp
Handle.GET("/villains/anakin-skywalker", () =>
{
  var resp = new Response()
  {
    StatusCode = 302,
    StatusDescription = "Moved Permanently"
  };
  resp.Headers["Location"] = "/villains/darth-vader";
  return resp;
});
```

### API for Redirection

To redirect your app's URL handler to another handler, call `GET /sc/redirect/{PortNumber}{FromUri};{ToUri}` on system port (by default 8181).

For example, to create redirection of `/` to `/myapp` on port 8080, the following handler should be called: `GET /sc/redirect/8080/;/myapp` on system port (by default 8181).

To create redirection on the startup of your application, you may call `Http.GET(8181, "/sc/redirect/8080/;/myapp");`. Keep in mind, that any other app can overwrite this redirect because there can only be one app handling the host root.

## Uri Aliasing

### URI Aliasing in Network Gateway

The classical example is to alias root URI to some other URI in your application. URI aliases are configured in `scnetworkgateway.xml` in your server directory.\
The following section is an excerpt from gateway configuration:

```markup
<UriAliases>
    <UriAlias>
        <HttpMethod>GET</HttpMethod>
        <FromUri>/</FromUri>
        <ToUri>/index.html</ToUri>
        <Port>8181</Port>
    </UriAlias>

    <UriAlias>
        <HttpMethod>GET</HttpMethod>
        <FromUri>/</FromUri>
        <ToUri>/launcher</ToUri>
        <Port>8080</Port>
    </UriAlias>
</UriAliases>
```

* **HttpMethod**: for which HTTP method the alias is created.
* **FromUri**: URI that should be aliased.
* **ToUri**: URI to which the aliased URI should be changed.
* **Port**: port on which this rule should apply.

In the example above we have declared two URI aliases:\
1\. On port 8181 from URI "/" to "/index.html". This is often used to display application initial "index" page when user just specifies the DNS name of the Web-server in the browser.\
2\. On port 8080 from URI "/" to "/launcher". Used to display a dynamic launcher page for the same purposes.

### API for URI Aliasing

Starcounter allows adding, modifying, listing, and deleting URI aliases using the following REST API (should be called on system port (by default 8181)):

* Adding and modifying a specific URI alias is done using `PUT /sc/alias`. The URI alias info should be in HTTP body with the following format:

```javascript
{
  "HttpMethod": "",
  "FromUri": "",
  "ToUri": "",
  "Port": 0
}
```

for example, `{"HttpMethod":"GET","FromUri":"/","ToUri":"/launcher","Port":8080}`

* Listing all URI aliases is done using `GET /sc/alias`:

```javascript
{  
  "Items":[  
    {  
      "HttpMethod": "GET",
      "FromUri": "/",
      "ToUri": "/launcher",
      "Port": 8080
    },
    {  
      "HttpMethod": "GET",
      "FromUri": "/myapp",
      "ToUri": "/myappalias",
      "Port": 8080
    }
  ]
}
```

Retrieving info about specific URI alias is done using `GET /sc/alias/{?}/{?}/{?}` where parameters are `string httpMethod, long dbport, string fromUri`.\
For example, `http://127.0.0.1:8181/sc/alias/GET/8080//SomeFromUri`

Deleting specific URI alias is done using `DELETE /sc/alias/{?}/{?}/{?}` where parameters are `string httpMethod, long dbport, string fromUri`.\
For example, `http://127.0.0.1:8181/sc/alias/GET/8080//SomeFromUri`

{% hint style="info" %}
In comparison with redirects, URI aliasing API adds entries to network gateway configuration and preserved upon gateway/code-host restart.
{% endhint %}


# Network Gateway

## Introduction

The network gateway is a key network communication component of Starcounter. The `scnetworkgateway.exe` process, which is separated from the database, represents the network gateway. It handles all external communications with the code-host, as well as communication between different code-hosts on one machine.

The gateway process talks to the codehost process (sccode) through shared memory. The same way codehost process is also separated from database process (scdata) and talks to it through shared memory. There is no direct connection between gateway and scdata, only through sccode. Network gateway parses network traffic, and prepares "messages" that are delivered to sccode through shared memory and executed there. The responses from sccode go back through the same shared memory to gateway, which sends them outside.

Detailed network gateway configuration and statistics can be retrieved using `GET /gw/stats` on system port (by default 8181).

## Configuration

Network gateway is configured in `scnetworkgateway.xml`, which is located in the server repository (e.g User/Documents/Starcounter/Personal). Here are some notable configuration options:

### WorkersNumber

Is the number of gateway worker threads. Normally this value should be 1-2. On high loads, this value can be increased.

### MaxConnectionsPerWorker

Is the maximum number of connection per each gateway worker. Connections are equally distributed between workers.

### MaximumReceiveContentLength

Is the maximum size of incoming HTTP body/content, in bytes. Requests with bigger bodies are rejected with `413 Request Entity Too Large` and closure of TCP connection.

### InactiveConnectionTimeout

Is the inactive HTTP connections life time in seconds. Inactive connections are those on which send/receive are not performed.

## Big size Uploads

Since the network gateway supports limited size uploads, the user has to write a custom big-data uploader for both HTTP and WebSocket protocols.

## Reverse Proxy Functionality

Starcounter gateway provides basic reverse proxy functionality based on HTTP Host header in requests. Reverse proxies are defined in `ReverseProxies` section of gateway configuration. Here is an example of reverse proxy that redirects all incoming HTTP requests on port 80, with Host header equals "[www.example1.sc](http://www.example1.sc)", to service on localhost and port 8080:

```markup
<ReverseProxy>
  <DestinationIP>127.0.0.1</DestinationIP>
  <DestinationPort>8080</DestinationPort>
  <StarcounterProxyPort>80</StarcounterProxyPort>
  <MatchingHost>www.example1.sc</MatchingHost>
</ReverseProxy>
```

* **DestinationIP**: IP address of destination server, to which proxied requests are redirected.

  or
* **DestinationDNS**: DNS name of destination server, to which proxied requests are redirected.
* **DestinationPort**: port address of destination service.
* **StarcounterProxyPort**: Starcounter gateway port on which requests that should be proxied are received.
* **MatchingHost**: HTTP Host header value to filter out requests to be processed by this proxy.

In the example above when HTTP request with `Host` header `www.example1.sc` comes on port 80 its automatically transferred to port 8080 on localhost.

You can see current configuration for reverse proxies by calling `GET /gw/stats` on system port.

To apply current gateway configuration (URI aliases and reverse proxy configuration) from `scnetworkgateway.xml` one should call `GET /gw/updateconf` on system port. If errors occur applying new changes - the previous configuration is kept.


# Static File Server

## Introduction

Starcounter comes with a built-in HTTP static file server intended for client-side files such as HTML, JavaScript and images. Each application can have several static resources directories that are creating one virtual resources hierarchy.

Static content is always cached in RAM, so you should not use it for video or other very large pieces of content. An external server should be used for such files. The cache is updated when you replace the files on disk.

## Directories Added by Convention

When your application start, Starcounter support exposing static resources by convention.

For example, having this:

```
C:\Users\JohnDoe\apps\app1>dir
app1.exe
wwwroot
```

and starting the application doing this

```
C:\Users\JohnDoe\apps\app1>star app1
```

will add **wwwroot** as a resource directory explicitly.

Please note the *working directory* is what will be used to qualify the full path, so if you are in some other directory starting the application, you will have to specify the directory explicitly. See below.

The same convention applies in Visual Studio. If you have a Starcounter application project and run it from there, the working directory of the project is used to find a **wwwroot** and if such folder exist, it will registered as a static resource directory.

## Application Specific Static Files

Application specific static files should be put in a directory with the name of the application. For example, for the [People](https://github.com/StarcounterApps/People) app, the HTML, CSS, images, JavaScript and other files that are strictly specific to that app should be put in the directory `wwwroot/People`. This prevents namespace collisions between apps running together.

## External Dependencies

External dependencies, such as fonts, Web Components and other files that might be used in more than one app should be put in a `sys` directory. For example, the [People](https://github.com/StarcounterApps/People) app puts the Polymer elements it uses in `wwwroot/sys`.

Starcounter has certain files that are served by default to the `sys` directory which makes them available to all apps. One example is Polymer; even if it's not put in the `sys` folder of individual apps, it's still available with the HTML import `<link rel="import" href="/sys/polymer/polymer.html">` because it's served by Starcounter.\
\
With this approach, we ensure that HTML imports only load each dependency one time.

For further instructions on how to add external dependencies to Web Components, read [avoiding loading the same files multiple times](/2.3.1/guides/web-apps/introduction-to-web-components#avoiding-loading-the-same-files-multiple-times) on the Web Components page.

## Specifying Static Resources Directory

There are several ways to explicitly add static resource directories to your application.

### Working Directory

When creating a new Starcounter application project, the default value for the projects *working directory* will be set to the project root directory. Hence, adding a **wwwroot** directory to the project will automatically make that a resource directory by means of the conventions described above.

To be even more specific, you can specify resource directories in the "Command line arguments" under the `Project | Properties | Debug` tab.

Command line arguments:

```
--resourcedir=thirdpartyscripts
```

By doing this, Starcounter will **add** that directory as an additional resource directory, in addition to a possible **wwwroot** as described above. You can specify both a relative directory or a fully qualified one. If you specify a relative directory, its qualified by means of the working directory, just as convention-based directories are.

### Star.exe CLI

When starting your application using `star.exe`, the resources directory can be set for that application explicitly using parameter `--resourcedir`, for example:

```
star --resourcedir=C:\MyWebsite\Content c:\MyWebsite\PMail.exe
```

You can use fully qualified paths, or relative paths. Relative paths are resolved by using the directory from where `star.exe` is invoked (i.e. the working directory).

### Code

Resources directory can also be added programmatically in your code, by calling `AppsBootstrapper.AddStaticFileDirectory`. The first parameter is relative/absolute static resources directory and second optional parameter is a port on which files should be served, for example:

```csharp
AppsBootstrapper.AddStaticFileDirectory("C:\\MyWebsite\\Content", 80);
```

To use `AddStaticFileDirectory`, add the `Starcounter.Apps.JsonPatch` assembly as a reference in the project. By default, it's found at `C:\Program Files\Starcounter\Public Assemblies`.

### Specifying Multiple Resource Directories

In both command-line mode aswell as in Visual Studio, we have shown how to specify a resource directory using the `--resourceDir` option. You can also use this option to specify multiple resource directories.

Specify multiple resource directories using ";" (semi-colon) as the separator, like this:

```bash
star --resourcedir=thirdpartyfiles;images;d:\mywebserver\htmlutils app.exe
```

Mixing relative and absolute paths work in harmony. Relative paths are resolved as has been described above.

### Constraints

You can specify only directories that exist. As a consequence, you can not specify paths including characters not supported by the underlying platform. For example, specifying this on Windows

```
star --resourcedir="foo>>>" myapp.exe
```

result in an error like this

> ScErrBadArguments (SCERR1001): One or more arguments was invalid. Parameter --resourcedir contains an illegal element: 'foo>>>', error: Illegal characters in path.

## Directory Locking

Once any file in the resource directory is served using HTTP - the parent directory can not be removed or renamed. This happens due to file changes observing mechanisms locking it. To be able to rename/delete/move the resource directory - the codehost has to be stopped for that operation.


# External HTTP Calls

## Introduction

Starcounter `Http` and `Node` classes represent a convenient way to communicate with other Starcounter instances as well as 3rd party HTTP servers. You can use `Http` and `Node` from within Starcounter programs or as a stand alone packages for generic .NET programs (as a generic HTTP client). When used in server Starcounter applications, `Http` and `Node` are suited for communicating **outside** the current codehost, for example, some Web-site or other codehosts. To communicate **within** current codehost, `Self` should be used.

## Why Not Use Existing HTTP APIs?

One reason why you would use `Http` and `Node` instead of using another HTTP client API is that `Http` and `Node` are really REST clients rather than HTTP clients. Another reason is their simplicity for REST style programming as they are normally used as a single line, non-verbose statement.

## How to Use HTTP

A convenient way to use synchronous or asynchronous HTTP calls is to use the `Http` class and its functions `GET`, `POST`, `PUT`, `PATCH` and `DELETE`. To use these calls synchronously makes using HTTP requests very similar to using regular function calls.

## How to Use Node

Instead of using static function calls, you can instantiate a representation of an external HTTP server. This is recommended when performance is critical as this method allows Node to avoid the overhead of parsing the host from the URL and ensures optimal connection reuse. Using the static `Http` class still reuses connections and caches Node instances internally, but it comes at a slight overhead and you run into the risk of the Node instances being disposed and recreated more times than necessary.

### Constructing a Node Instance

The Node constructor accepts the following parameters: the DNS host name of the server we are trying to communicate to, an optional port number, aggregation parameters and an optional receive timeout:

```csharp
Node(String hostName,
     UInt16 portNumber = 0,
     Int32 defaultReceiveTimeoutMs = 0,
     Boolean useAggregation = false,
     UInt16 aggrPortNumber = 0)
```

for example:

```csharp
Node localNode = new Node("www.starcounter.com");
Node localNode2 = new Node("127.0.0.1", 8080);
Node localNode3 = new Node("buildserver", 8080);
```

### Prerequisites

To use Node you need to reference the assembly `Starcounter.Rest` (to use Request and Response add also a reference to *Starcounter.Internal*)

**NOTE**: To use Node outside of Starcounter in 32-bit software you should reference the same libraries in the `32BitComponents` subfolder in pointed to by the environment variable `StarcounterBin`, since the 32-bit libraries are not placed in the Windows GAC.

**NOTE**: Node instances are thread unsafe. See *How Node works internally* section for more details.

### Using Node Methods

Node supports most popular HTTP methods: GET, POST, PUT, DELETE. User can specify arbitrary HTTP method as well.

For example, one of the Node REST **asynchronous** GET call has the following signature:

```csharp
void GET(String uri,
         String customHeaders,
         Object userObject,
         Action<Response, Object> userDelegate,
         Int32 receiveTimeoutMs = 0)
```

where:

* **uri** - **relative** resource URI, e.g.: "/", "/index.html", "/chairs", etc.
* **customHeaders** - user custom headers separated by "\r\n", or null if no custom headers needed, e.g.: "MyHeaderName: value123\r\n", "MyHeaderName1: value123\r\nMyHeaderName2: value456\r\n".
* **userObject** - user object to be passed to delegate or `null` if not needed.
* **userDelegate** - user delegate to call once the HTTP response is obtained.
* **receiveTimeoutMs** - timeout on receive.

In case if Node method fails it will construct erroneous `Response` object with non-successful status code and error information in `Body`, and then call user delegate with that object.

If exception occurs within user delegate, its logged to Starcounter server log if `Node.ShouldLogErrors` is set to `True`, otherwise exception is simply re-thrown.

Another version of Node REST **synchronous** GET simply returns the HTTP response instead of calling user delegate:

```csharp
Response GET(String uri, String customHeaders, Int32 receiveTimeoutMs = 0)
```

Returned `Response` should never be null.

Other standard HTTP method calls (PUT, POST, DELETE) have an optional body parameter, for example:

```csharp
void PUT(String uri,
         String body,
         String customHeaders,
         Object userObject,
         Action<Response, Object> userDelegate,
         Int32 receiveTimeoutMs = 0)
```

which also has a corresponding version of function that returns HTTP response instead of delegate.

In order to user arbitrary user HTTP method you can call the following Node call:

```csharp
void CustomRESTRequest(String method,
       String uri,
       String body,
       String customHeaders,
       Object userObject,
       Action<Response, Object> userDelegate,
       Int32 receiveTimeoutMs = 0)
```

You can obtain some information about Node instance using the following methods:

* **UInt16 PortNumber** - port number for this Node instance.
* **String HostName** - host name for this Node instance.
* **Uri BaseAddress** - URI of this Node.

### More on User Delegate

User delegate described above accepts two parameters: received `Response` object and object supplied by user. When delegate exits `Node` checks if response should be returned to the original `Request`. User indicates that by setting the `Response` object property on original `Request` object, like in the following example:

```csharp
Handle.GET("/postponed", (Request req) =>
{
    Http.POST("http://www.mywebsite.com/echotest", "Here we go!", null, null, (Response resp, Object userObject) =>
    {
        // Modifying the response object by injecting some data.
        resp.Headers["MySuperHeader"] = "Here is my header value!";
        resp.Headers["Set-Cookie"] = resp.Headers["Set-Cookie"] + ";MySuperCookie=CookieValue";
        req.Response = resp;
    }); // "resp" object will be automatically sent when delegate exits.

    return HandlerStatus.Handled;
});
```

### Error Checking

Once the Node call is finished, user should check for status code of the HTTP response (returned Response object should never be null).

The code below demonstrates the redirection of root HTTP URI "/" to startup HTML page `http://www.mywebsite.com/index.html`:

```csharp
// Redirecting root to index.html.
GET("/", (Request req) =>
{
    // Doing another request to obtain static file response.
    Response resp = Http.GET("http://www.mywebsite.com/index.html");

    // Checking that response status is correct.
    if (!resp.IsSuccessStatusCode)
    {
        throw new Exception(@"REST call returned
            status code: " + resp.StatusCode);
    }
    // Returns this response to original request.
    return resp;
});
```

### How Node Works Internally

During the first request, Node instance creates a TCP connection with specified server. All subsequent synchronous calls on this Node instance will use this TCP connection. Asynchronous Node calls are using pool of connections. If the connection is dropped - its automatically recreated. If connection can not be re-established, special error Response object is created (with `StatusCode` 503). One connection(socket) is used for synchronous Node calls because data should arrive in order (with multiple connections client data can arrive out-of-order). The network connection resources are automatically cleaned up upon Node instance garbage collection.

## Setting Receive Timeout

User can specify receive timeout both in synchronous and asynchronous `Node` and `Http` modes. Timeout is specified in milliseconds as last parameter for each method (GET, POST, PUT, etc), for example:

```csharp
Response GET(String uri, Int32 receiveTimeoutMs = 0)
```

Receive timeout can also be specified on Node instance when constructing the Node (or at any time using accessor `DefaultReceiveTimeoutMs`), and its value will be used as default for all calls on this Node instance, unless receive timeout is overwritten for a particular call, as described above.

Default value 0 means infinite timeout.\
If timeout is reached and response is not received yet, the erroneous `Response` object is constructed as explained before.


# WebSocket

## Introduction

WebSocket is a web technology providing full-duplex communications channels over a single TCP connection.

> Learn more about WebSocket on Wikipedia

Starcounter WebSocket implementation is based on RFC 6455.

WebSocket connection upgrade can be made inside an ordinary HTTP handler by calling `SendUpgrade` method on received HTTP `Request` object, for example:

```csharp
Handle.GET(8080, "/wstest", (Request req) =>
{
    // Checking if its a WebSocket upgrade request.
    if (req.WebSocketUpgrade)
    {
        // Setting some headers and cookies on response for WebSockets upgrade.
        List<String> myCookies = new List<String>
        {
            "MyCookie1=123", "MyCookie2=456"
        };

        var myHeaders = new Dictionary<String, String>()
        {
            { "MyHeader", "MyHeaderData" }
        };

        // Performing upgrade and getting WebSocket object
        // (SendUpgrade call implicitly sends an HTTP response confirming
        // WebSocket upgrade, so another response can't be returned
        // in this handler).
        WebSocket ws = req.SendUpgrade("echotestws", myCookies, myHeaders);

        // Immediately sending a message on the obtained WebSocket.
        ws.Send("Hello WebSocket!");

        // Sending another message on the WebSocket.
        ws.Send("Hello again WebSocket!");

        // Indicating that response on the original request was already
        // sent (during SendUpgrade call).
        return HandlerStatus.Handled;
    }

    // We only support WebSockets upgrades in this HTTP handler
    // and not other ordinary HTTP requests.
    return new Response()
    {
        StatusCode = 500,
        StatusDescription = "WebSocket upgrade on " + req.Uri + " was not approved."
    };
});
```

Once the `SendUpgrade` is called and WebSocket object is available, it can immediately be used to perform sends and other operations allowed on WebSocket.

WebSocket can be identified as a one UInt64 integer, that user can store in database and by some other means. Whenever needed, user can restore WebSocket object using the ID and perform operations on that object.\
To obtain a WebSocket ID, call `WebSocket.ToUInt64()`, in opposite, to create a WebSocket object just pass the UInt64 ID to WebSocket constructor:

```csharp
WebSocket ws = new WebSocket(savedWsId);
ws.Send("My server message!");
```

## Details of the SendUpgrade Call

When the `SendUpgrade` method is called, the approving WebSocket upgrade HTTP Response is sent immediately. To be able to get the WebSocket ID before calling `SendUpgrade`, call the `UInt64 Request.GetWebSocketId()`.

The WebSocket `SendUpgrade` method on Request has the following signature:

```csharp
WebSocket SendUpgrade(
    String groupName,
    List<String> cookies = null,
    Dictionary<String, String> headers = null,
    IAppsSession session = null)
```

where:

* `groupName`: string identifying WebSocket "group" on which subsequent events for this WebSocket should arrive.
* `cookies`: Cookies that should be set on returned WebSocket upgrade HTTP response.
* `headers`: Custom headers that should be set on returned WebSocket upgrade HTTP response.
* `session`: a session object that is attached to this WebSocket in subsequent events.

Once the WebSocket object is returned, user can fetch the ID representing this WebSocket (`ToUInt64()`), and of course, perform data sends and disconnect.

## WebSocket Group Name and Disconnect Handlers

To register a specific WebSocket group to handle WebSocket data receiving events, the `Handle.WebSocket` method should be used, for example:

```csharp
Handle.WebSocket("echotestws", (String s, WebSocket ws) =>
{
    Console.WriteLine("Received on WebSocket: " + s);
    WebSocket.Current.Send("Here is the server push!");
    ws.Send("Here is the server push!"); // Does the same as previous line.   
});

Handle.WebSocket("echotestws", (Byte[] s, WebSocket ws) =>
{
    ws.Send(s);
});
```

As the name suggests, the group name is used to group together certain types of WebSockets, for example, chat messages, log events, game objects updates, etc.

Note that if arriving frame type is `Text` and the only registered handler is for `Binary` then WebSocket connection will be closed with a type: `WS_CLOSE_CANT_ACCEPT_DATA`. Same applies for vice-versa scenario.

To handle WebSocket disconnect event, the `Handle.WebSocketDisconnect` method should be used, for example:

```csharp
Handle.WebSocketDisconnect("echotestws", (WebSocket ws) =>
{
    // Handle resources associated with WebSocket ws.
});
```

Disconnect event is triggered when underlying socket connection is closed, regardless the reason for it (e.g. normal WebSocket closure, connection drop, etc). Note that WebSockets have no inactive timeout disconnect.

## WebSocket Object

The returned `WebSocket` object contains the following notable methods:

Sending string data on WebSocket:

```csharp
void Send(String data,
  Boolean isText = true,
  Response.ConnectionFlags connFlags = Response.ConnectionFlags.NoSpecialFlags)
```

Sending binary data on WebSocket:

```csharp
void Send(Byte[] data,
  Boolean isText = false,
  Response.ConnectionFlags connFlags = Response.ConnectionFlags.NoSpecialFlags)
```

Using `isText` parameter you can specify if your WebSocket data should be sent as `Text` or `Binary` frame (see RFC6455)

`Response.ConnectionFlags` parameter is used to manipulate corresponding WebSocket connection: for example, if you want to close the connection with client or send data and then close the connection. Please see section about handling responses for more information on `Response.ConnectionFlags`.

NOTE: When doing operations on the same WebSocket but from different Starcounter schedulers - the order in which operations (like `Send`) will actually be performed is not guaranteed to be the same as the order in which they were initiated.

Disconnecting an active WebSocket (with an error message and/or close code according to RFC6455):

```csharp
void Disconnect(String message = null,
  WebSocketCloseCodes code = WebSocketCloseCodes.WS_CLOSE_NORMAL)
```

Other following methods and properties are available:

* `UInt64 ToUInt64()`: identifier that represents the WebSocket, normally used for storage in user code.
* `IAppsSession Session`: session object that was initially attached on WebSocket if any.
* `static WebSocket Current`: static object identifying currently active WebSocket.
* `enum WebSocketCloseCodes`: list of available WebSocket disconnect codes, according to RFC6455.
* `Boolean IsDead()`: checks if WebSocket is already disconnected or invalid.

## Operating on Multiple WebSockets

Sometimes its needed to perform the some "action" on a group of WebSocket connections, for example, to unicast some common data. To accomplish this, user can create a WebSocket object from saved UInt64 ID and perform operations with that object (send/disconnect). The only requirement is that operating thread should be attached to Starcounter scheduler. The following code snippet demonstrates broadcasting of previously saved WebSockets every second from arbitrary .NET thread:

```csharp
[Database]
public class WebSocketId
{
    // WebSocket identifier.
    public UInt64 Id { get; set; }

    // Number of messages sent on this WebSocket.
    public UInt32 NumBroadcasted { get; set; }
}

...
WebSocketSessionsTimer = new Timer((state) =>
{
    // Getting sessions for current scheduler.
    Scheduling.ScheduleTask(() =>
    {
        Db.Transact(() =>
        {
            // Going through all active WebSockets.
            var result = Db.SQL("SELECT w FROM WebSocketId w");
            foreach (WebSocketId wsDb in result)
            {
                WebSocket ws = new WebSocket(wsDb.Id);
                // Checking if ready to disconnect
                // after certain amount of sends.
                if (wsDb.NumBroadcasted == 100)
                {
                    // Disconnecting this WebSocket.
                    ws.Disconnect();
                    wsDb.Delete();

                    // Proceeding to next active WebSocket.
                    continue;
                }
                String sendMsg = "Broadcasting id: " + wsDb.Id;
                // Sending message to this WebSocket.
                ws.Send(sendMsg);
                wsDb.NumBroadcasted++;
            }
        });
    });
}, null, 1000, 1000);
```

## Additional Information

`Session` object contains the reference to the last active WebSocket: `ActiveWebsocket`, which is associated with this session. `ActiveWebsocket` however can be `null` when session has no WebSocket attached to it.


# Avoiding URI conflicts

## Introduction

There are some HTTP "namespacing" precautions that you need to take to make your app run safely alongside other apps.

## Namespacing dynamic HTTP handlers

Your app should only create HTTP handlers (using `Handle.GET`, `Handle.POST`, etc) that begin with the app name. For example:

```csharp
Handle.GET("/myapp", ()
{
  var page = new MyAppHomePage();
  return page;
});
```

By default it is not enforced. You might enforce it by enabling a database configuration flag: [`EnforceURINamespaces`](/2.3.1/guides/database/database-configuration).

## Namespacing static HTTP resources

The above principle should also be used for the static resources that are exposed from the `wwwroot` subdirectory in your app.

All the files that are unique to your app (HTML templates, CSS stylesheets, custom JavaScript) should be placed under `wwwroot/<appname>`.

For example a file saved as `wwwroot/myapp/style.css` will be exposed by the static HTTP server as <http://localhost:8080/myapp/style.css>. You don't have to worry that some other app will overwrite `style.css`, because it will use it's own directory for the file.

In contrast to the above, all the files that are NOT unique to your app (shared libraries, Custom Elements HTML Imports, Bower components) should be placed under `wwwroot/sys`. This allows apps to be sufficient yet allow for resource sharing.

For example, there might be two applications that use `<link rel="import" href="/sys/paper-input/paper-input.html">`. Both of the apps host the static resource `http://localhost:8080/sys/paper-input/paper-input.html`. When requesting this URL, you are served by the last app that was started. Because the URLs are the same, there is no conflict of two Web Components loaded simultaneously. More about that in [Using Web Components](/2.3.1/guides/web-apps/introduction-to-web-components#using-web-components).

The next pages explain what's the benefit of all that - data sharing and screen sharing.


# TCP Sockets

## Introduction

TCP is a connection-oriented protocol.

## Using TCP Sockets

TCP sockets in user code are represented by objects of the class `TcpSocket`. User code can define a handler for retrieving data and detecting disconnects on a socket by using `Handle.Tcp` method:

```csharp
static void Handle.Tcp(UInt16 port, Action<TcpSocket tcpSocket, Byte[] data> handler);
```

* `port`: local port on which incoming TCP connections should be accepted.
* `tcpSocket`: active socket object. This object can be used to send data, disconnect socket, etc. When incoming `data` is `null` it means that socket was disconnected.
* `data`: incoming data on TCP socket.

`TcpSocket` class has the following notable methods:

* `UInt64 ToUInt64()`: identifier that represents the TCP socket, normally used for storage in user code.
* `void Disconnect()`: disconnects an active TCP socket.
* `void Send(Byte[] data)`: sends given data on active TCP socket.
* `Boolean IsDead()`: checks if TCP socket object is still active.
* `static TcpSocket Current`: currently active TCP socket for this scheduler.

NOTE: When doing operations on the same TcpSocket but from different Starcounter schedulers - the order in which operations (like `Send`) will actually be performed is not guaranteed to be the same as the order in which they were initiated.

Once the TCP socket object is returned, user can fetch the ID representing this socket (ToUInt64()), and of course, perform data sends and disconnect.

In the following example we process TCP sockects on port 8585 and send echo data back. The example shows how socket ID can be used to associate socket with resources, for example objects in database:

```csharp
Handle.Tcp(8585, (TcpSocket tcpSocket, Byte[] incomingData) =>
{
    UInt64 socketId = tcpSocket.ToUInt64();
    // Checking if we have socket disconnect here.
    if (null == incomingData)
    {
        // One can use "socketId" to dispose resources associated with this socket.
        return;
    }

    // One can check if there are any resources associated with "socketId" and otherwise create them.
    // Db.SQL("...");

    // Sending the echo back.
    tcpSocket.Send(incomingData);

  // Or even more data, if its needed: tcpSocket.Send(someMoreData);
});
```


# UDP Sockets

## Introduction

UDP is a connection-less protocol with unreliable delivery.

## Using UDP Sockets

Incoming UDP datagrams with maximum size of 30000 bytes are supported (user code can send datagrams up to 65000 bytes). In order to receive UDP datagrams, user needs to register a handler using `Handle.Udp`:

```csharp
void Udp(UInt16 port,
         Action<IPAddress clientAddr,
         UInt16 clientPort,
         Byte[] datagram> handler)
```

* `port`: local port on which to listen for datagrams.
* `clientAddr`: represents client's IP address.
* `clientPort`: holds client port number.
* `datagram`: incoming UDP datagram.

In order to send a UDP datagram from user code, static method `UdpSocket.Send` is used:

```csharp
static void Send(IPAddress ipTo,
                 UInt16 portTo,
                 UInt16 portFrom,
                 Byte[] datagram)
```

* `ipTo`: destination IP address.
* `portTo`: destination port.
* `portFrom`: local port from which UDP datagram should be sent (usually same port on which `Handle.Udp` was called).
* `datagram`: the actual UDP datagram that should be sent.

In the following example we receive (on port 8787) and send the same echo UDP datagram back to the client:

```csharp
Handle.Udp(8787, (IPAddress clientIp, UInt16 clientPort, Byte[] datagram) =>
{    
    // One can update any resources associated with "clientIp" and "clientPort".
    UdpSocket.Send(clientIp, clientPort, 8787, datagram);
});
```


# Publishing Apps

## Introduction

If your app is blendable, it might be used in complex UI systems consisting of multiple apps coming from different vendors.

To make sure that apps play well together, we recommend that you follow certain conventions.

These conventions exist for two reasons:

1. to prevent conflicts between apps in the same database
2. to provide a common ground, expectable user experience

## Mapping File

Each app that have blendable views should have a `<AppName>.map.md` file which contains the list of them.

The file should be provided at the root of the App Warehouse package. If the app is open source, the file should be provided in the code repository, preferably at the root.

Every view must be explained in the file and preferably illustrated with a screenshot.

In the future, the information in this file is used for automated blending as well as for [testing purposes](https://github.com/Starcounter/Guidelines/issues/26).

Sample `Images.map.md` ([source](https://github.com/StarcounterApps/Images/blob/develop/Images.map.md)):

```
# Blendable views

## /images/partials/contents/{[Simplified.Ring1.Content](https://github.com/StarcounterApps/Simplified/blob/master/Ring1/Content.cs)}

Shows a simple page for `Content` preview, image or video. In case of unexisting
content, shows empty file preview image.

![screenshot](docs/screenshot-content.png)
```

## Entry Point URI

Every blendable web app should have the entry point URI that follows the pattern `/<appname>`, e.g. `/kitchensink` (lowercase).

When you access this URI, the app main screen should be presented.

If there is no main screen, show a splash screen saying that the app is running.

Sample `MainHandlers.cs` ([source](https://github.com/StarcounterApps/People/blob/94341b2dc62ad6637808313c367f986a417d349b/src/People/Api/MainHandlers.cs#L32-L35)):

```csharp
Handle.GET("/people", () =>
{
    return Self.GET("/people/organizations");
});
```

## UI Metadata

By convention, a blendable web app must provide a metadata JSON that responds to the `app-name` blending token.

System apps and navigation apps like [Launchpad](https://github.com/StarcounterApps/Launchpad) will display a link to the entry point URI using the app name from this metadata. They will also use the icon, if provided.

The response JSON can consist of the following properties:

| Property      | Explanation                                               |
| ------------- | --------------------------------------------------------- |
| `name`        | **Required.** Human readable app name.                    |
| `description` | Optional. Short (single sentence) description of the app. |
| `html`        | Optional. URI to a view that contains the app icon.       |

Sample `MainHandlers.cs` ([source](https://github.com/StarcounterApps/CssVariablesManager/blob/develop/src/CssVariablesManager/Api/MainHandlers.cs#L13)):

```csharp
Handle.GET("/cssvariablesmanager/app-name", () => new AppName());
```

Sample `BlendingHooks.cs` ([source](https://github.com/StarcounterApps/CssVariablesManager/blob/4aad5186522a6297bc4ca7f713d91424220d6552/src/CssVariablesManager/Api/BlendingHooks.cs#L9)):

```csharp
Blender.MapUri("/cssvariablesmanager/app-name", "app-name");
```

Sample `AppName.json` ([source](https://github.com/StarcounterApps/CssVariablesManager/blob/4aad5186522a6297bc4ca7f713d91424220d6552/src/CssVariablesManager/ViewModels/AppName.json)):

```javascript
{
  "name": "CssVariablesManager",
  "description": "CssVariablesManager",
  "html": "/cssvariablesmanager/AppIcon.html"
}
```

### App Icon

The icon is an image (preferably inline SVG) in the HTML file which path is provided as the `html` property in the `app-name` metadata.

Check out the [Icons](/2.3.1/cookbook/icons) cookbook to learn more about icons.

Sample `AppIcon.html` ([source](https://github.com/StarcounterApps/CssVariablesManager/blob/4aad5186522a6297bc4ca7f713d91424220d6552/src/CssVariablesManager/wwwroot/CssVariablesManager/AppIcon.html)):

```markup
<template>
    <svg slot="cssvariablesmanager/app-icon" viewBox="0 0 185 185" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
        <use xlink:href="/cssvariablesmanager/images/cssvariablesmanager.svg#css-file" />
    </svg>
    <template is="declarative-shadow-dom">
        <slot name="cssvariablesmanager/app-icon"></slot>
    </template>
</template>
```

## Required and Sample Data

The app should not create required nor sample data without the user's consent.

### Required Data

If the app requires any data to operate (for example a list of the world's countries in the `Country` table), it should present a nagging information with a button that creates the data:

* on the top of the main page
* possibly in every view that needs the required data

### Sample Data

If the app offers some nice-to-have sample data, it should offer a button that creates this data:

* on the main page, in a welcome message that is dismissible once (not per user)
* on the [Settings page](/2.3.1/guides/publishing#settings-page)

## Settings Page

If your app is configurable, it's good for to provide a user interface for changing the app settings.

By having a settings page mapped to the common token, your configuration UI can appear along settings pages from other apps. This is good because it gives the end user a single go-to place to configure all apps.

The common pattern is to have the settings page addressable by `/<appname>/settings`, blended to the token `settings`.

The settings page might contain a button to do a "Factory Reset" of the app (restores required data) or a "Sample data" button that populates the database with sample data.

Sample and prefab apps that have a settings page:

* <https://github.com/StarcounterApps/Products>
* <https://github.com/StarcounterApps/SignIn>

## Submitting to the App Warehouse

To make your app discoverable by others, you're invited to publish it in the App Warehouse.

Submitting apps to the App Warehouse requires an invitation.

[Contact Starcounter](https://starcounter.com/about/) for more information.


# Working with Starcounter

## Introduction

This section describes tools that make it easy to develop and run applications in Starcounter.


# Release Channels

## Introduction

Starcounter supports a number of different release [channels](http://downloads.starcounter.com/download). We use these channels to roll out updates to users, starting with our close to hourly builds, all the way up to our Release channel releases.

## Release

Stability: **High**, Frequency: **Low**

This channel contains releases that have been thoroughly tested by the Starcounter team and our clients. The latest release on this channel is used for the master branch of our [sample apps](https://github.com/starcounterapps). For most purposes, using the latest release from this channel is recommended. It's updated approximately once every six months.

## Release Candidate

Stability: **Medium**, Frequency: **High**

This channel is for versions that are stable enough to be considered for release. It's updated daily with bug fixes. We don't introduce new functionality in the Release Candidate, that's done on the Develop channel.

## Develop

Stability: **Low**, Frequency: **High**

This channel is the unstable iterated release channel. It's updated every night after a full day of work. Features from the Develop release are most probable to appear in the next “Release” version, use it to try out these features. Since the versions from this channel are under constant development, they should be used with care - expect that they will corrupt your database and contain bugs that won't be fixed immediately.

## Custom

Stability: **Varied**, Frequency: **Varied**

This channel is for custom builds that are used for specific purposes. It may, for example, help to detect problems in an environment. Releases of this channel should mainly be used by developers who are explicitly directed to them by our support.

## Long-Term Support (LTS)

Stability: **High**, Frequency: **Low-Medium**

This channel is updated nightly when there are hotfixes and patches for the latest release of the Release channel. There will not be a major release in this channel unless there is also a release in the Release channel, thus, this channel and the Release channel does not differ in functionality.


# Starting and Stopping Apps

## Introduction

Starcounter apps can be started and stopped with the `star` CLI, the Administrator, and Visual Studio. This page describes how to use these tools to start and stop apps.

## Starting

Starcounter applications are compiled into `.exe` files. These applications are then run inside Starcounter.

Applications in Starcounter are always run in the context of a single database. This means that accessing information in that database is lightning fast. It does not mean that the application is not allowed to access other databases (more on node-to-node communication later).

### Command Line

To start an app from the command line, you use the `star` command. It accepts .cs files in addition to .exe files.

```bash
C:\> star hello.exe
[Starting "hello.exe" in "Default" on "Personal" (127.0.0.1:8181)]
"hello.exe" started in database "Default". Default port is 8080 (Executable),
8181 (Admin))

C:\> star test.cs
[Starting "test.cs" in "Default" on "Personal" (127.0.0.1:8181)]
"test.cs" started in database "Default". Default port is 8080 (Executable),
8181 (Admin))
```

The command line option `--database` (or `-d`) can be used to specify what database to run the application in.

```bash
C:\> star --database=mydb hello.cs
[Starting "hello.cs" in "mydb" on "Personal" (127.0.0.1:8181)]
"hello.cs" started in database "mydb". Default port is 8080 (Executable),
8181 (Admin))
```

### Administrator Web Interface

In the default Starcounter configuration, the Administrator web interface is available at <http://localhost:8181/>.

Go to <http://localhost:8181/>. Select a database from the "Databases" menu (e.g. `default`) or create a new database, then click on "Start Executable". Provide path to the .exe or select it from the disk and press "Start".

### Visual Studio

In Visual Studio, press F5 to start the application. The Starcounter personal server will load the executable and execute it in the Default database context. If the personal server is not running it will be started automatically.

#### Specifying Options in Visual Studio

In fact, Visual Studio plugin supports most of the `star.exe` command line arguments. You can specify particular arguments by altering `Project | Properties | Debug | Command line arguments`. By default, the application is started in a database `default`. To start the application in any other database, such as `foo`, add this to `Command line arguments`: `-d=foo`.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5d029d0-8680-11e7-9944-1f85270462c6%2Fcbbdaef6-d55f-11e5-933e-f1c8dda21a12.png?generation=1503327413178330\&alt=media)

## Stopping

Contrary to desktop programs, the program remains loaded after main has been executed. You can unload all code without stopping the database. The database memory lives in a separate process and the database memory is shared between you program code and the database process.

### Command Line

In order to unload a program, you need to run the `star --stop <application>` command.

```bash
C:\> star --stop hello.cs
[Stopping "hello.cs" in "Default" on "Personal" (127.0.0.1:8181)]
  - Restarting database "Default"
Stopped "hello.cs" in database "Default"
```

### Administrator Web Interface

In the default Starcounter configuration, the Administrator web interface is available at <http://localhost:8181/>.

Go to <http://localhost:8181/>. Select a database from the "Databases" menu (e.g. `default`). You will see a list of apps running in the selected database. You can use the "Stop" button to unload each of the apps.

### Visual Studio

You cannot stop an app from Visual Studio.

## Running Multiple Apps

When you run multiple apps in the same database, as with `hello.exe` and `test.cs` in the example below, they are all loaded into the same database process. The code you load keeps adding to the code being loaded into the database.

Starting apps for the first time, even if you have other apps loaded, is fast. The reason for this is that there is no need to recycle the host process. On the other hand, if you want to restart an application, there is one thing that you have to keep in mind: restarting an app requires the host process to be recycled. The consequence of this is that all other apps that are running need to be stopped, unloaded, and started again. Due to this, it takes significantly more time to restart an application when there are other apps running than when you start an application.

For example, if you have three apps up and running and you would like to start another app, let's name it `app4.exe`. Then you would simply run `star app4.exe` which would start the app without much delay. Now, if you want to **restart** two of the other apps, `app1.exe` and `app2.exe` the situation would be different. Simply running `star app1.exe` would require the host process to be recycled, `app2.exe`, `app3.exe`, and `app4.exe` to be stopped, unloaded, and started again. The same would be the case when we restart `app2.exe`. This code describes the gist of the process:

```python
host = [app1, app2, app3, app4]

def star(my_app):
  if my_app in host:
    new_host = []
    for app in host:
      stop(app)
      unload(app)
      start(app)
      new_host.append(app)
    return new_host

  #starts app and adds it to the already running host process
  else:
    start(my_app)
    return host

host = star(app2)
```

To handle this in cases where you would like to restart a set of applications you can run staradmin stop host and then start the applications one at a time. This allows us to circumvent the inefficient `for` loop and start the application directly. If you have to restart several applications, this is the preferred approach.

In other cases, where you just want to start an application from scratch or restart one or two applications in a small set of apps, you should simply start, or restart the desired application by running star myapp.exe. Starcounter will, in this case, make sure that the state of your applications is maintained and that all applications that were running previously are restarted. This may take some more time, as explained above.

All running Starcounter applications must have a unique name. You cannot run two instances of the same application in a single database.

## Reading the Console Output

The console output does not go to the standard output (i.e. console). After all, your program is likely to run in the cloud or on your local personal server. Instead, any standard output is directed to a server side memory buffer. The Administrator web user interface can monitor the output. If there is no Administrator web session listening to the output, the output is discarded.

In the default Starcounter configuration, the Administrator web interface is available at <http://localhost:8181/>.

To see the console output in Administrator, navigate to the database by clicking on its name (e.g., `default`) in the list of databases or navigate to the application by clicking on the application name in the list of applications.


# Administrator Web UI

## Introduction

Administrator is a web based administration tool that comes with Starcounter. It can be started in three different ways:\
1\. Run `staradmin start server`\
2\. Run an app\
3\. Execute `scservice.exe` - the `Starcounter Personal Server` shortcut that is added to the desktop after installation does this

After starting the Administrator, it's accessible from `localhost:8181`.

The default port `8181` can be changed during installation or in the server configuration.

## Handling Databases

Databases are handled from `localhost:8181/#/databases`.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe59fa3f0-8680-11e7-9944-1f85270462c6%2F1.png?generation=1503327413854494\&alt=media)

### Default Database

Starcounter creates a default database if an application is started and there is no existing database. Apps in the default database are available at port `8080` by default.

### Create and Delete Databases

It's possible to create and delete databases in the Administrator. To prevent conflicts when running several parallel databases, their ports have to be different. The port can be specified under the "Advanced" options when creating a new database.

Create new databases at `localhost:8181/#/databases` by pressing "New database".

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe59fcb00-8680-11e7-9944-1f85270462c6%2F3.png?generation=1503327411377299\&alt=media)

Databases are also deleted in the same view. Deletions have to be verified by entering the name of the database in the pop-up window.

### Start and Stop Databases

Start and stop databases by clicking the "Start" or "Stop" buttons at `localhost:8181/#/databases`:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe59fcb01-8680-11e7-9944-1f85270462c6%2F56.png?generation=1503327411281226\&alt=media)

Databases with a green checkmark are running.

## View Application Output

Go to a database by clicking on its name in the list of databases. Then, click on the application name in the "Applications" list. This will open up a view that displays the output of the application.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe59ff210-8680-11e7-9944-1f85270462c6%2Fappoutput2.gif?generation=1503327412539433\&alt=media)

## Control Apps

Apps will start together with the database if "Auto-Start" is clicked. The padlock icon shows if an app can be deleted.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe59ff211-8680-11e7-9944-1f85270462c6%2FDatabase.png?generation=1503327412739918\&alt=media)

## SQL Browser

### Execute SQL Queries

The data of a database can be queried in the SQL browser. See [SQL reference](/2.3.1/guides/sql) for details on the syntax.

The queries that are supported by the method `Db.SQL` are also supported except that [literals](/2.3.1/guides/sql/literals) are used in the SQL browser instead of variables.

The app that defines that table needs to run in order to query it.

For example, the class `Person` can be queried this way:

```sql
SELECT Person.FullName, Text FROM Quote WHERE Person.FirstName = 'Albert'
```

This is the result:

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a08e50-8680-11e7-9944-1f85270462c6%2FScreenshot-2015-10-02-17.23.40.png?generation=1503327412628857\&alt=media)

### SQL Query Plan

If you navigate to "Query Plan" after the SQL request, you can see the steps to access the data.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a0dc70-8680-11e7-9944-1f85270462c6%2F5.png?generation=1503327413735311\&alt=media)

## App Warehouse

Apps can be downloaded to a database from the `App Warehouse` tab. Once an app has been downloaded, it can be started and stopped from its database page.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a0dc71-8680-11e7-9944-1f85270462c6%2FAppStoreTab.png?generation=1503327413783128\&alt=media)

### Download and Install Apps

Click the `Download` button to download an app. Downloaded apps can be controlled on the database page.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a10380-8680-11e7-9944-1f85270462c6%2FAppstore1.png?generation=1503327411075154\&alt=media)

## Starting Executables <a href="#starting-executables" id="starting-executables"></a>

You can launch an application in a database by going to `localhost:8181/#/databases/default/executabeStart`. Specify the path to your `.exe` application in the field. 

![](broken://files/-KxJXlZICXtS7zpAVSOi)

## Database Configuration

Access the database settings by going to `localhost:8181/#/databases/default/settings`. The available settings are:

* Database port - 8080 by default
* Scheduler count, defines the degree of parallelization - the default value is the number of available logical CPU cores. The max number of schedulers is 31. Running two code hosts with 16 schedulers each will fail since it's more than 31. &#x20;
* Chunks, for advanced users, should not be modified for most databases - 65536 by default

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a151a0-8680-11e7-9944-1f85270462c6%2F7.png?generation=1503327412884434\&alt=media)

## Log

Go to `localhost:8181/#/server/log` to see debug steps, notices, warnings, and errors. Sort the log by "Source" to see the behavior of specific components.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a178b0-8680-11e7-9944-1f85270462c6%2F8.png?generation=1503327413042646\&alt=media)

## Network

Go to `localhost:8181/#/server/network` to see internal environment, the information about network facilities for Starcounter installation that comes with the network gateway.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5a19fc0-8680-11e7-9944-1f85270462c6%2F9.png?generation=1503327410916449\&alt=media)

## Server Configuration

The system port and gateway port can be changed at `localhost:8181/#/server/settings`.

![](broken://files/-L24xPRKvCfIHl6xOjoY)

The "Allow Remote Access" option determines if the Administrator accepts requests from outside localhost. If it's set to "yes", any other machine in the same network can access the Administrator and if it's "no", then the Administrator will only accept requests from localhost. The default value is "no".&#x20;


# Star CLI

## Introduction

The `star` command line interface (CLI) handles tasks in a database. This page covers the most important commands. The rest of the commands can be found with `star --help` or `star --helpextended`.

## Starcounter Version

The version of the current Starcounter installation is displayed with `star --version` or the shorthand `star -v`.

```bash
> star --version
Version=2.3.1.7018
```

## Starting Apps

Apps are started with `star [app]` by specifying the path to the `exe` file. For example:

```bash
~\Apps\HelloWorld> star .\src\HelloWorld\bin\Debug\HelloWorld
```

### Set Resource Directory

When starting an app, Starcounter will fail to find the application resource directory (`wwwroot`) if the current working directory is not the root of the project or the same directory as the resource directory. This will, for apps that use the Starcounter web stack presented in [Blendable Web Apps](/2.3.1/guides/web-apps), throw, `ArgumentOutOfRangeException` when opening the app.

To solve this, specify the resource directory with the `--resourcedir` option:

```bash
~\Apps\HelloWorld\src\HelloWorld\bin\Debug> star --resourcedir=../../wwwroot HelloWorld
HelloWorld -> default (started, default port 8080, admin 8181)
```

Read more about this on the page [Static File Server](/2.3.1/guides/network/static-file-server).

### Specify Database

By default, apps are started in the `default` database. To start apps in another database, use the `--database` option:

```bash
> staradmin new db myDatabase
Created (Name=myDatabase)
> star --database=myDatabase HelloWorld
HelloWorld -> mydatabase (started, default port 8080, admin 8181)
```

The shorthand notation is `-d`: `star -d=myDatabase HelloWorld`.

### Change App Name

To start an app with another name than the existing one, use the `--name` option:

```bash
> star --name=HelloUniverse HelloWorld
HelloUniverse -> default (started, default port 8080, admin 8181)
```

## Stopping Apps

Apps are stopped with the `--stop` option:

```bash
> star HelloWorld
HelloWorld -> default (started, default port 8080, admin 8181)
> star --stop HelloWorld
HelloWorld <- default (stopped)
```

Read [Starting and Stopping Apps](/2.3.1/guides/working-with-starcounter/starting-and-stopping-apps) for more information.


# StarAdmin CLI

## Introduction

With **staradmin.exe**, users can interact with Starcounter from the command-line, running different management tasks. These tasks include stopping of running applications, killing Starcounter processes, unloading and reloading databases, and more.

## Basic Use

Running staradmin without arguments will display the usage message in the console. The general syntax of the staradmin tool is:

```
staradmin [options] command [<command options>] [<parameters>]
```

The **options** section include options that apply either to the program itself or to the majority of the commands. Options are specified using a `--` prefix and include both flags and properties. As an example, the `--help` flag tells staradmin to write out the usage message, while the `--database=<name>` property allow a user to specify a specific database the upcoming command is to target. Shorthand `-d=<name>` has the same effect.

The **command** tells staradmin what is to be done. Commands are usually verbs. Common commands include `stop`, `list` and `help`.

Some commands support **command options**. As their name implies, these options are optional and specific for the given command. As an example, the `list` command support the `max=<n>` property, allowing the user to limit the set of entries in a displayed list to the value `<n>`.

Finally, most commands will require or at least support some **command parameters**. The parameter will usually describe a type of object the command should operate on; examples include `stop app <name>` and `help list`. The first one instruct staradmin to stop an *application* by name; the second that it should display help on the *list* command.

## Getting Help

To see the latest help overview, run `staradmin --help`.

To find extended help on a certain command or a known topic, issue `staradmin help <command|topic>`, for example `staradmin help stop` to see the help for the *stop* command.

## Console Command

The **console** command shows the console output from applications

```
staradmin console [<databases>]
```

Running `staradmin console` without parameters shows the console output from the default database.

Running `staradmin -d=foo console` shows console output from the "foo" database.

Provide a space-separated list of database names as command parameters to show output from multiple databases. For example, `staradmin console foo` shows console output from the "foo" database. `staradmin console foo bar baz` shows console output from the "foo", "bar" and "baz" databases.

## Delete Command

The **delete** command deletes various types of objects, e.g. databases. Usage:

```
staradmin delete [--force] [--failmissing]
```

Example: delete database&#x20;

To create a user-specified database use

```bash
C:\>staradmin --database=NewDbName delete db
```

### Command Options

The delete command supports the `--force` flag. This flag tell staradmin you don't want to confirm the requested delete, which otherwise is the default in case you are deleting some sensitive artifact such as a database. Use this flag with care, there is no going back.

```
staradmin -d=foo delete --force db
```

The `--failmissing` flag toogle how staradmin behaves when the artifact you want to delete is not found. By default, such case is treated as a successful operation. With this flag applied, `staradmin` will instead issue an error.

```
staradmin -d=nonExisting delete --failmissing db

ScErrDatabaseNotFound (SCERR10002): ScErrDatabaseNotFound (SCERR10002):
A database with the specified name was not found.
```

### Object Types

The delete command supports the following type of objects to be deleted.

* **Databases**. Usage: `staradmin -d=default delete db`. Deletes a database.

## Kill Command

The **kill** command kills processes relating to Starcounter. Usage:

```
staradmin kill <target>
```

Use `all` as the command parameter to target killing all processes relating to Starcounter on the current machine. Use this option with care and make sure no mission-critical processes are running.

## List Command

The **list** command provides viewing of lists. It takes the general form:

```
staradmin list <type>
```

where *type* will indicate the kind of list you want to see. To see a list of databases, use the **db** type; to see a list of all running applications, use **app**.

### Command Options

The list command supports the `max=<n>` property. By using this property, you tell staradmin not to list more entries than the value of `<n>`.

```
staradmin list --max=10 app
```

*Lists running applications, limiting the result to a maximum of 10*

### Object Types

The list command supports the following type of objects to be listed.

* **Databases**. Usage:  `staradmin list db`. List all databases part of the current installation, even those that are not running.
* **Applications**. Usage:  `staradmin list apps`. List all applications currently running, including information on the database they are running in.
* **Logs**. Usage:  `staradmin list log`. Shows the content of the server error log. See more usage on the [Error log](/2.3.1/guides/working-with-starcounter/error-log) page.

## New Command

The **new** command allows creation of new artifacts. It takes the general form:

```
staradmin new <type>
```

where *type* specifies the kind of artifact to create. To create a database, use the **db** type; for applications, use **app**.

**Example: create database**

To create a user-specified database use

```bash
C:\>staradmin new db foo
```

### Object Types

The new command allow the following type of artifacts to be created:

* **Databases**. Usage:  `staradmin new db foo`. Creates a new database named "foo".
* **Applications**. Usage:  `staradmin new app`. Creates a new application source code file, normally "app.cs".

### Set Configuration Options on Creation

All the available configuration options in the underlying REST JSON representation can be set from the command line. These are:

* `Uri`
* `DataDirectory`
* `TempDirectory`
* `DefaultUserHttpPort`
* `FirstObjectID`
* `LastObjectID`

The options are specified on the creation of the database using this syntax:

```
staradmin new db DataDirectory=C:\Users\Per\Foo DefaultUserHttpPort=1234 Uri=http://example.com/api/databases/foo/configuration"
```

## Reload Command

The **reload** command reloads data into a data source, usually a database. Usage:

```
staradmin reload [source] [--file=<path>]
```

If no *source* is given, `db` is used as the default.

### Object Types

* **Databases**. Usage: `staradmin reload db`. Reloads a database.

### Command Options

The reload command supports the `--file=<path>` option. The filename is resolved to the same directory from which the command runs. If the file option is omitted, the default file is used.

```
staradmin -d=bar reload db --file=data.sql
```

*Reloads the "data.sql" file into the "bar" database.*

## Start Command

The **start** command is used to start processes. It takes the general form:

```
staradmin start <type>
```

where *type* specifies what should be started. Use  `staradmin start db` to start the default database; use `staradmin start server` to start the Starcounter server.

Example: start database

To start a user-specified database use

```bash
C:\>staradmin --database=UserDbName start db
```

To start the application with .exe extension on a specified database use:

```bash
C:\[path to your application]>star --database=newdb YourApplicationName.exe
```

To find out more about how to start and stop applications read this article.\
The console output does not go to the console, but directed to a server side memory buffer and it is possible to view it from the Administrator Web UI.

**NOTE**: Starting databases or the server like this is not the normal scenario. Instead, Starcounter employs a design where processes are started on demand. When starting applications, using `star <app>` or from within Visual Studio, doing "Start" on a **Starcounter Application project**, these processes are started for you automatically (if not already running).

*Starting the "foo" database*

```
staradmin -d=foo start db
Starting foo (started, code host PID: 6048)
```

### Object Types

The start command support starting the following types.

* **Databases**. Usage:  `staradmin start db`. Starting the default database, including all its support processes.
* **The server**. Usage: `staradmin start server`. Starts the Starcounter server.

## Stop Command

The **stop** command is used to stop running processes or applications. It takes the general form:

```
staradmin stop <type> [reference]
```

where *type* will indicate the type of the given reference. As an example, to stop an application with the name *foo*, use `staradmin stop app foo`. To stop a database named *bar*, use `staradmin -d=bar stop db`. To stop the default database, use `staradmin stop db`.

Example: stop database

To start a user-specified database use

```bash
C:\>staradmin --database=UserDbName stop db
```

### Object Types

The stop command supports stopping the following types.

* **Databases**. Usage:  `staradmin stop db`. Stops the default database, including all its support processes, effectively freeing all database memory.
* **Applications**. Usage:  `staradmin stop app [name]`. Stops an application by name. The `-d` option can be used to tell staradmin in what database look for it; if not given, the default database is assumed.
* **Hosts**. Usage:  `staradmin stop host`. Stops the code host of the default database. Support processes stay resident, meaning that database memory is not freed.

## Unload Command

The **unload** command unloads data from a data source, usually a database. Usage:

```
staradmin unload [source] [--file=<path>]
```

If no *source* is given, `db` is used as the default.

### Object types

* **Databases**. Usage: `staradmin unload db`. Unloads a database.

### Command Options

The unload command supports the `--file=<path>` option. The filename is resolved to the same directory from which the command runs. If the file option is omitted, the default file is used.

```
staradmin -d=bar unload db --file=data.sql
```

*Unloads the "bar" database into the "data.sql" file.*

The `--allowPartial` option unloads the database, allowing the unload to be partial. Usage: `staradmin unload db --allowPartial`.

The `--shiftKey` option makes every key being unloaded increase with the given number.

```
staradmin unload db --shiftKey=99999
```

*Makes every key being unloaded increase by 99999.*

## Exceptions

### ScErrHandlerAlreadyRegistered (SCERR13003)

This exception is thrown when two identical handlers are registered on the same port. A common way to hit this is by running two databases where both ports are set to 8080, which is the default port. In its simplest form, it looks like this:

```
> staradmin new db default
Created (Name=default)                                                                                                  > staradmin new db another
Created (Name=another)                                                                                                  > staradmin start db default
Starting default (started, code host PID: 9300)                                                                         > staradmin start db another

System.Exception: ScErrHandlerAlreadyRegistered (SCERR13003): This handler has already been registered. Can't register URI handler "GET /sc/htmlmerger?{?}" on port 8080. This handler is already registered.
```

The solution is to start the databases on different ports:

```
> staradmin new db default
Created (Name=default)                                                                                                  > staradmin new db another DefaultUserHttpPort=5000
Created (Name=another)   
```


# StarDump CLI

## Introduction

StarDump CLI is a command-line interface that unloads and reloads database data. It's found in `C:\Program Files\Starcounter\StarDump`. &#x20;

The data is unloaded into `sqlite3` files that can be examined or [modified](/2.3.1/guides/working-with-starcounter/unload-reload#modify-the-schema-in-the-database) with [DB Browser for SQLite](http://sqlitebrowser.org/).

{% hint style="info" %}
All examples on this page assume that you have `C:\Program Files\Starcounter\StarDump` as an environment variable.
{% endhint %}

The general format for the StarDump CLI looks like this:

```bash
stardump [command] [options]
```

There are two commands, `unload` and `reload`, and both of them have their own options.

## Unload Database

The `unload` command takes a Starcounter database and creates a `sqlite3` file with the data. Running `stardump unload` without any options will unload the `default` database to `%TEMP%\stardump-<database>-<date>.sqlite3`. To specify the unload further, these options can be used:

| Name                           | Type       | Default value                               | Description                                                                              |
| ------------------------------ | ---------- | ------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `-db`, `--database`            | `string`   | `default`                                   | Database name to unload                                                                  |
| `-f`, `--file`                 | `string`   | `%TEMP%\stardump-<database>-<date>.sqlite3` | Output file path with file name                                                          |
| `-b`, `--buffersize`           | `int`      | `500`                                       | Number of rows in a single `INSERT`operation                                             |
| `-scp`, `--skipcolumnprefixes` | `string[]` | `__`                                        | Column prefixes to skip                                                                  |
| `-stp`, `--skiptableprefixes`  | `string[]` |                                             | Table prefixes to skip                                                                   |
| `-st`, `--skiptables`          | `string[]` |                                             | Table names to skip                                                                      |
| `-ut`, `--unloadtables`        | `string[]` |                                             | Table names to unload                                                                    |
| `-V`, `--verbose`              | `int`      | `2`                                         | Verbose mode, higher number prints more \[Error:0, Warning:1, Notice:2, Info:3, Debug:4] |

The option values are case sensitive. Also,  `unloadtables`disables `skiptableprefixes` and `unloadtables` if they are used together. `string[]` values should be space or comma separated.

Example of unload:

```
stardump unload --database default --file C:\Temp\default.sqlite3
```

The targeted database can either be running or stopped when it's unloaded. If the database is running, StarDump creates a snapshot of the database when the unload starts so that no changes made during the unload are included.&#x20;

You can use `stardump unload --help` to get more information about the available options.

## Reload Database

The `reload` command takes a `sqlite3` file and reloads it into a Starcounter database. The basic structure of a reload looks like this:

```
stardump reload --database [DatabaseName] --file [FilePath]
```

The `--database` and `--file` option always have to be specified.&#x20;

These are all options for the `reload` command:

| Name                                | Type     | Description                                                                                                                                                                                                   |
| ----------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-db`, `--database`                 | `string` | Database name to reload into                                                                                                                                                                                  |
| `-f`, `--file`                      | `string` | Path to `sqlite3` file to reload                                                                                                                                                                              |
| `-fr`, `--forcereload`              | none     | Force reload even if the database already contains data. The user has to take care of object ID uniqueness.                                                                                                   |
| `-offset`, `--insertobjectnooffset` | `long`   | `ObjectNo` offset in the target database                                                                                                                                                                      |
| `-reassign`, `--reassignobjectno`   | none     | Sets the `ObjectNo` offset to the highest`ObjectNo` in the target database `-noschema`, `--skipschema`                                                                                                        |
| `-noschema, --skipschema`           | none     | If flag is set, then schema creation and validation will be skipped.   In `--skipschema` mode, StarDump reloads data of matching tables and columns between existing Starcounter database and reloading dump. |
| `-V`, `--verbose`                   | `int`    | Verbose mode, higher number prints more \[Error:0, Warning:1, Notice:2, Info:3, Debug:4]                                                                                                                      |

None of these are have a default value except for `--verbose` which is set to `2`.

If `--forcereload` is not used, the database should be dropped and created prior to reload to ensure that the database is empty:

```
staradmin -d=default delete --force db
staradmin -d=default new db DefaultUserHttpPort=8080
stardump reload --database default --file C:\Temp\default.sqlite3
```

You can use `stardump reload--help` to get more information about the available options.


# Working in Visual Studio

## Introduction

Developing Starcounter applications is straightforward and not tied to any certain development environment. Applications targeting .NET are normally built using Visual Studio, but would work using just `msbuild` or even the C# compiler too.

In Visual Studio, applications can be built using a [Starcounter Visual Studio extension](https://marketplace.visualstudio.com/items?itemName=Starcounter.StarcounterforVisualStudio), or just using standard Visual Studio templates such as the C# "Console Application" project template.

## Using the Visual Studio Extension

Starcounter provides a Visual Studio extension to simplify the development of Starcounter applications. It provide templates to build on and the ability to start applications from Visual Studio. The extension can either be installed during the installation process of Starcounter by choosing that as an option, or by manually downloading and installing it from the [Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=Starcounter.StarcounterforVisualStudio).

### Templates

There are currently two project templates and three item templates included in the extension.

**Project templates**:

The project templates are used to scaffold projects that target Starcounter. When instantiated, the project will have some default Starcounter-specific *assembly references* set up and include some build hooks that will aid in building. For more information on the specifics, see section about [creating applications using standard templates](/2.3.1#create-a-starcounter-application-using-standard-templates) later in this text.

Project templates include:

* Starcounter Application
* Starcounter Class Library

**Item templates**:

* Starcounter HTML template with dom-bind
* Starcounter Typed JSON
* Starcounter Typed JSON with code-behind

#### Starcounter Application Template

The Starcounter application template is the starting point to creating applications with Starcounter. It contains four references: `Starcounter`, `Starcounter.Internal`, `Starcounter.Logging`, and `Starcounter.XSON`. Additionally, it comes with a boilerplate `Program.cs` file that looks like this:

```csharp
using System;
using Starcounter;

namespace MyApp
{
    class Program
    {
        static void Main()
        {

        }
    }
}
```

#### Starcounter Class Library

The Starcounter class library template is the starting point for creating a shared data model to use across applications. For example, the [Simplified](https://github.com/StarcounterApps/Simplified) DLL that is used to provide a shared data model to the Starcounter [sample apps](https://github.com/StarcounterApps) is built with this template. It contains the same references as the Starcounter application template. This is how the boilerplate `Program.cs` file looks:

```csharp
using System;
using Starcounter;

namespace MyClassLibrary
{

    [Database]
    public class Entity1
    {
        public string Field1 { get; set; }
    }
}
```

#### Starcounter HTML Template with dom-bind

This item templates gives a starting point for creating HTML view definitions with Polymer.

It contains the following code:

```markup
<link rel="import" href="/sys/polymer/polymer.html">

<template>
    <template is="dom-bind">

    </template>
</template>
```

#### Starcounter Typed JSON

This template is the starting point for creating a view-model definition using JSON-by-example. It is simply an empty `.json` file containing an empty JSON object:

```javascript
{
}
```

#### Starcounter Typed JSON with code-behind

This template is the same as the Starcounter Typed JSON file, except that it also provides a code-behind file. Thus, two files are created with this template, `.json` and `.json.cs`.

The `.json` file is identical to the Starcounter Type JSON file. The `.json.cs` file contains the following code:

```csharp
using Starcounter;

namespace MyApp
{
    partial class MyPage : Json
    {
    }
}
```

## Starting Applications From Visual Studio

With the Visual Studio Extension, apps can be started directly from the development environment. This is done the same way any other application would be started from Visual Studio, by clicking the `Start` button or f5.

Further instructions on starting applications in Visual Studio can be found in [Starting and Stopping Apps](/2.3.1/guides/working-with-starcounter/starting-and-stopping-apps). There it is also described how it is possible to set particular arguments on application start from `Debug` -> `MyApp Properties` -> `Debug`.

## Using Standard Visual Studio

As was established in the introduction, the Visual Studio extension (or even Visual Studio) is not a requirement for building and running Starcounter applications. You could very well create applications using just a text editor and msbuild, or even the compiler.

In this section, we'll see how to work with applications using just standard Visual Studio.

### Create a Starcounter App with Standard Templates

Using the standard C# "Console Application" project, we can turn that into a proper Starcounter application with these simple steps:

1. Create the project, name it for example "Hello".
2. Edit the `Hello.csproj`
   1. Add references to Starcounter assemblies.
   2. Add an import to the Starcounter .targets file.

`Hello.csproj` (snippet showing additions)

```markup
<ItemGroup>
  <Reference Include="Starcounter, Version=2.0.0.0, Culture=neutral, PublicKeyToken=d2df1e81d0ca3abf">
    <Private>False</Private>
  </Reference>
  <Reference Include="Starcounter.Internal, Version=2.0.0.0, Culture=neutral, PublicKeyToken=d2df1e81d0ca3abf">
    <Private>False</Private>
  </Reference>
  <Reference Include="Starcounter.Logging, Version=2.0.0.0, Culture=neutral, PublicKeyToken=d2df1e81d0ca3abf">
    <Private>False</Private>
  </Reference>
  <Reference Include="Starcounter.XSON, Version=2.0.0.0, Culture=neutral, PublicKeyToken=d2df1e81d0ca3abf">
    <Private>False</Private>
  </Reference>
</ItemGroup>

<Import Project="$(StarcounterBin)\Starcounter.MsBuild.targets" />
```

Without the extension, you can't start the application project simply from within Visual Studio. Luckily, it's easily done using tooling being part of Starcounter.

1. Open a command line prompt.
2. CD to the directory of your built application.
3. Run `star Hello.exe`.

## Exceptions

### ScErrAppDeployedEditionLibrary (SCERR12019)

Starcounter throws `SCERR12019` if you add a Starcounter assembly, such as `Starcounter.Extensions` that's found in
&#x20;`C:\Program Files\Starcounter\Public Assemblies`. To fix this, change the `Copy Local` property of the added assembly to `false`.

###


# Error Log

## Introduction

The Starcounter **error log** contains detailed **information about warnings and errors**. Whenever an error occurs, and whenever a component of Starcounter needs to issue a warning or a notice, a detailed message with the relevant severity is written to the error log.

## How to Access the Error Log

### Using Administrator Web UI

You can access the error log via the Administrator Web UI. To view the error log on your machine:

1. Open a browser
2. Navigate to <http://localhost:8181/#/server/log>.

### Using Command-Line Tools

There are two ways to view the content of the log using the command-line tools: via `staradmin.exe` and via `star.exe`.

### Using StarAdmin CLI

For greatest flexibility, use `staradmin.exe`. Type `staradmin list --max=50 log` to see the 50 most recent log entries.

By default, `staradmin list log` will show you the 25 latest logged entries with severity `Notice` and higher (i.e. including `Warning` and `Error`, but not `Debug`).

Examples. To see,

1. all entries (`Notice` and up), type `staradmin list --max=all log`.
2. the 100 latest entries, including `Debug`, type `staradmin log --max=100 debug`.
3. all entries (`Notice` and up), type `staradmin list --max=all log`.
4. the 100 latest entries, including `Debug`, type `staradmin list --max=100 log debug`.
5. all errors, type `staradmin list --max=all log errors`
6. the 20 latest errors and warnings, type `staradmin list --max=20 log warnings`.
7. the 50 latest entries, independent of their severity, logged by the "Starcounter" log source, type `staradmin --max=50 list log all Starcounter`.

Finally, entries displayed by `staradmin list log` can be filtered by a named database by applying the global `-d` option. For example, to see the 10 last entries from the `Starcounter.Host` source, logging from within the `default` database, type `staradmin -d=default list --max=10 log all Starcounter.Host`.

### Using star CLI

You can instruct `star.exe` to display log entries scoped to the operation star executes, for example starting an application. You do this using the new **--logs** option.

Typing `star --logs app.exe` will first start "app.exe" and then output all log entries that was written to the log from the time when the command was invoked. By default, logs with `Notice` and up is displayed. To see debug logs too, instead type `star --verbose --logs app.exe`.

### Viewing the Raw File

You can also see the direct content of the log by browsing the file content on your hard drive. To do this,

1. Locate the root server directory of your Starcounter installation. The default path for the personal server is: `C:\Users\[YourName]\Documents\Starcounter\Personal`.
   * In older versions of Starcounter, it was: `C:\Users\[YourName]\Documents\Starcounter\[Version]\Personal`
2. Open the `Logs` subdirectory.

The log data is in one or more files named with the convention `Starcounter.[nnnnnnnnnn].log` where \[nnnnnnnnnn] is an opaque sequence number used by Starcounter. If you see several files, the one with the highest number contains the most recent entries.

## Writing to the Error Log

Not only Starcounter components can write to the log. Applications running in Starcounter can do so too. Writing to the log is done using the `LogSource` class, part of the `Starcounter.Logging` namespace.

```csharp
using Starcounter;
using Starcounter.Logging;

class Program
{
    static void Main()
    {
        new LogSource("PerSamuelsson").LogWarning("I dont do any good!");
    }
}
```

If you run the above application using `star app.cs`, viewing the logged entry can be done with the `staradmin list log` command, using `staradmin list log all PerSamuelsson`.

The `LogSource` class has support for logging errors (critical, text, and from exceptions), warnings and notices. Invoking any of the corresponding `LogSource` methods, such as `LogWarning` above, will assure the message ends up in the log.

### Diagnostic Logging

In addition to errors, warnings and notices, Starcounter also allows diagnostic logging using the `Debug` and `Trace` methods respectively. Logging using the `Debug` method will only be available in Starcounter versions built with the DEBUG configuration. Similarly, logging using the `Trace` method will only be available in Starcounter versions built with the TRACE configuration.

### Conditional Low-Level Trace Logging

Various Starcounter components also support low-level diagnostics in Starcounter TRACE builds by enabling *trace logging*. With trace logging turned on, trace messages emitted by the Starcounter runtime is routed to the log using the `Debug` severity. Trace logging is an experimental feature and should not be considered future compatible. It is driven by an environment variable, `SC_ENABLE_TRACE_LOGGING`. To set this flag and thereby effectively enable trace logging for a set of Starcounter components, make sure all Starcounter processes are stopped and apply the `--tracelogging` flag to `scservice.exe`.

```csharp
staradmin kill all
start scservice --tracelogging
```


# Using HTTPS on NGINX

## Introduction

Secure Socket Layer (SSL) is a protocol that offers security through encryption for communications between client and server. The encryption process is made possible through the use of digital certificates verified by a third party Certificate Authority and as we know it is the most commonly implemented in the HTTPS combination protocol.

The SSL protocol aims to provide solutions to two simple security problems:

1. Secure data transmission between the `client` and `NGINX`.
2. Obtain proof that an involved party is the one we want to grant access to the encrypted data.

All input traffic passes through SSL to NGINX in private and integral way, where it is being unwrapped and forwarded to the Starcounter server.

## Prerequisites

To get started you will need to ensure some basic things are on your server.\
Further, you also need to have the NGINX web server installed. Click here to chose your OS and then follow installation documentation.

We will generate 2 files: the Private-Key file for the decryption of your SSL Certificate, and a certificate signing request (CSR) file used to apply for your SSL Certificate. If you remember two problems we aim to avoid with SSL from above, now things should fall into place.

> It is highly important to set proper file permissions on private key file and NGINX server configuration file containing password to the handshake!

## Generate Private Key

You might have purchased a SSL certificate already from trusted SSL vendor.\
Otherwise you might be interested to use let's encrypt, a free certificate provider. It doesn't compromise security at all and website has a convenient tools that help you go through the process.\
Staging, testing and other non-production apps can use a free self-signed SSL certificate instead of purchasing one. In that case the connection will be encrypted however every time we try to access the corresponding website protected by the SSL certificate we will receive a warning that the certificate was not issued by trusted SSL vendor.

Generating SSL private key in your local environment will require you to install OpenSSL tool.\
Use openssl to generate a new private key.

```bash
$openssl genrsa -des3 -out /var/www/master.oops-email.pass.com.key
...
Enter pass phrase for master.oops-email.pass.com.key:
Verifying - Enter pass phrase for master.oops-email.pass.com.key:
```

Generated private key can stripped of its password so it can be loaded without manual password entry. You can do it if it's your test key.

```bash
$ openssl rsa -in /var/www/master.oops-email.pass.com.key -out /var/www/master.oops-email.com.key
```

## Generate CSR

Now that you've created your key, it's time to create the Certificate Signing Request, which will be used by the Certificate Authority of your choice to generate the Certificate that SSL will present to other parties during the handshake.

```bash
$ openssl req -x509 -new -key master.oops-email.com.key -out master.oops-email.com.csr
```

The result will be a master.oops-email.com.csr file in your local directory (alongside the master.oops-email.pass.com.key private key file from the above step).\
You will be asked to enter a set of information. Before we proceed let's see which information we actually stated in the above command:

You will be asked to enter some information about your location and company. The most important part is the Common Name field which should match the name that you want to use your certificate with - your domain name.\
Example of fill-in:

```bash
Country Name (2 letter code) [SE]:Sweden
State or Province Name (full name) [Some-State]:Stockholm
Locality Name (eg, city) []:Stockholm
Organization Name (eg, company) [Internet Widgits Pty Ltd]:YourName, Inc.
Organizational Unit Name (eg, section) []:YourUnitName
Common Name (e.g. server FQDN or YOUR name) []:your_domain.com
Email Address []:admin@your_domain.com
```

## Configure NGINX to Use SSL

We have created our key and certificate files under the NGINX configuration directory.\
The .key file is your private key, and should be kept secure. The .csr file is what you will send to the CA to request your SSL certificate.\
Now we just need to modify server configuration to use those by adjusting our server block file the way it is written below:

```bash
server {
listen 80;
server_name master.oops-email.com;
return 301 https://$host$request_uri;
}

server {
listen 443 ssl;
ssl_certificate     /var/www/master.oops-email.com.csr;
ssl_certificate_key /var/www/master.oops-email.com.key;
ssl on;

location / {
proxy_pass http://191.239.210.246:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; }

location /website/update {
root /var/www/master.oops-email.com;
}

location ~ /__(.*)/[A-Z0-9]+ {
                proxy_pass              http://191.239.210.246:8080;
                proxy_set_header        Host            $host;
                proxy_set_header        X-Real-IP       $remote_addr;
                proxy_set_header        Upgrade         $http_upgrade;
                proxy_set_header        Connection      "upgrade";
                proxy_read_timeout      86400;
        }
}
```

When you are done restart NGINX server. This should reload your configuration, now allowing it to respond to both HTTP and HTTPS (SSL) requests.\
Don't forget to test your setup, first using normal HTTP

```http
http://server_domain_or_IP
```

then using SSL to communicate

```http
https://server_domain_or_IP
```


# Using HTTPS on IIS

## Introduction

This article explains how to use Microsoft [Internet Information Server](https://www.iis.net/) (IIS) as an HTTPS proxy on top of Starcounter. While NGINX might be a better web server overall, it requires a separate Linux machine. IIS can be installed alongside Starcounter on the same Windows machine.

## Installing IIS

IIS is available from the `Turn Windows features on or off` console.

The following items should be installed.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe506bf50-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-0.png?generation=1503327413618978\&alt=media)

You can add any extra features if needed.

## Setting up IIS

Open Internet Information Services (IIS) Manager from the Start menu. And install the following components with `Get New Web Platform Components`.

* URL Rewrite 2.0
* Application Request Routing 3.0

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe506bf51-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-1.png?generation=1503327413630858\&alt=media)

Enable `system.webServer/webSocket` configuration section via Configuration Editor.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe506e660-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-2.png?generation=1503327411853430\&alt=media)

Restart the machine.

Open `Application Request Routing Cache` section and go to it's settings page.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe506e661-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-3.png?generation=1503327411914293\&alt=media)

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe506e662-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-4.png?generation=1503327412052791\&alt=media)

Enable proxy, update timeout to `600` seconds, and click the `Apply` button.

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe5070d70-8680-11e7-9944-1f85270462c6%2Fstarcounter-https-iis-5.png?generation=1503327412138231\&alt=media)

## Setting up IIS Website

By default there should be a `Default Web Site` item in the `Sites` section of IIS. The website should listen on the `80` port. Check that by opening `http://localhost/` in your browser. It should show the default IIS webpage.

Create and configure a new website if the default one is missing or you want to listen on another port rather than `80`.

[Create a self signed HTTPS certificate](https://technet.microsoft.com/en-us/library/cc753127\(v=ws.10\).aspx) and assign it to the website which should be used as a proxy.

Open `https://localhost/` in your browser and make sure that the default IIS webpage is also shown.

Open or create `web.config` file in the root folder of the website and update it's content to the following.

```markup
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <system.webServer>
        <webSocket enabled="true" receiveBufferLimit="4194304" />
        <rewrite>
            <rules>
                <clear />
                <rule name="HTTP -> HTTPS" enabled="true" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                        <add input="{HTTPS}" pattern="^OFF$" />
                    </conditions>
                    <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" redirectType="Temporary" />
                </rule>
                <rule name="80 -> 8080 port" patternSyntax="ECMAScript" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                        <add input="{HTTPS}" pattern="^OFF$" />
                        <add input="{CACHE_URL}" pattern="^(.+)[:][/][/]" />
                    </conditions>
                    <action type="Rewrite" url="{C:1}://localhost:8080/{R:0}" />
                </rule>
                <rule name="80 -> 8080 port, https">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
                        <add input="{HTTPS}" pattern="^ON$" />
                    </conditions>
                    <action type="Rewrite" url="http://localhost:8080/{R:0}" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>
```

**Note:** the rule names can be anything, but better keep it meaningful. The `8080` port should be updated with the port of the Starcounter database IIS should redirect to.

Make sure that Starcounter database is running and the proxy is working.

* `http://localhost/` should redirect to `https://localhost/`
* `https://localhost/` should open `http://localhost:8080/`


# Run Starcounter in Production

## Introduction

While Starcounter offers replication facilities for the enterprises to implement their high availability business scenarios (e.g., when a point-of-sales is disconnected from a main server, but continues to operate autonomously until the connection resumes), it doesn't provide a built-in support for disasters-related scenarios such as hot failover, regular backups and disaster recovery. A large variety of ready-made facilities are available today to support those disaster-related scenarios out of the box. E.g., most of the facilities needed for those are shipped with Windows Server 2012 R2 Standard off the shelf.

When it comes to running applications, developers want to perform the "full replication" of their apps, which means not only data itself, but also database configurations, static files, text log files, bulk data database files, user downloads etc. This goes beyond the scope of pure database replication for disaster recovery (like SQL Server AlwaysOn): either you try to store everything in the database to rely on its facilities for failover, or you run database facilities along with manual maintenance of failover machine. Instead of pure database facilities, it is possible to choose more general facility which will fit in most of use cases and works on the level of operating system.

Starcounter is not taking a goal to outperform those with any of our potential house-grown tools, preferring to focus on our core unique features. Instead, we do our best to supply our customers with the latest information on doing these scenarios with the OS standard facilities. This article aims to provide such information for community version users as well.

## Logs and Checkpoint Files

Starcounter is an in-memory technology, hence it only uses disks for persistence and recovery rather than a primary storage. Starcounter streams changes in your data to disk, while uses only primary memory to read data. Changes from committed transactions are logged to disk by a means of log writer and stored in Starcounter `.log` files. Periodically a disk image of the database is updated. This image, which is also known as a checkpoint, is stored in `.sci` and `.sci2` files. When checkpoint creation is executed, log files with records that the checkpoint now incorporates become redundant and thus are renamed from `.log` to `.log.bak` and are a subject of manual deletion. The fraction of `.log` files that are still in work along with snapshot image files together represent a current durable state of the database.

**Having** `.log.bak` **files regularly erased.** In order to clean `.log.bak` files you need to create a task in your preferred task scheduler for Windows or in Windows Task Scheduler to run `del` command on your `.log.bak` files. Please make sure that the created task in Task Scheduler do actually run as expected. Sometimes it will not run because the correct user privileges aren't set.

## Backup and Failover

Fault-tolerance is a property of a system to continue operating correctly after a software or a hardware failure. Fault-tolerance is achieved by redundancy on physical level together with some failover strategy. Failover strategies entail preventing malfunctions of hard drives (RAID, SAN), electricity chains problems (backup battery pack), general hardware failure (running multiple standby machines with different level of steadiness, running multiple active load-balanced machines).

Note that doing backups of data and providing failover strategy are not mutually exclusive. In general, you might end up with hot failover machine with a corrupted database if you corrupt your data within a human factor (e.g., uploading of a wrong application to production). Backups save you from human factor failure, failover is used for hardware factor failure.

While failover is about disaster recovery, scaling-out a system to multiple replicated nodes is done for high availability. Not only a hardware disaster may affect availability of a system, but also can do a deny of its service due to increased workloads. You might be familiar with situations when some database server "dies" under 1000 of simultaneous connections. Starcounter is highly available by design, so that a single node is capable of millions transactions per second out of the box. However, hardware issues might happen and you need to get ready for them. Different strategies for failover/disaster recovery are discussed in a checklist further.

## Database Backups

Starcounter works with data files on a byte level, so that some of these files are locked and other processes and users are secured from reading them. This locking prevents users and services from making inconsistent copies of the files that are currently under operation.

The simplest way to backup your data is a **cold backup**: stop Starcounter, wait when log writer finishes writing changes to disk, copy data files, start Starcounter. We do not recommend doing cold data backup unless you have a strong reason to "stop the world". A better, stop-less way to backup data is regularly available within all workstation and server editions of Windows operating system.

Windows [Volume Shadow Copy Service (VSS)](https://en.wikipedia.org/wiki/Shadow_Copy) allows making consistent copies of data files (known as VSS snapshots) even when the files are in use, and hence enables **hot backup** of your data. A VSS snapshot is a read-only point-in-time copy of a (disk) volume. Snapshots allow the creation of consistent backups of the volume, ensuring that the contents do not change and are not locked while the backup is being made. This is indeed an optimistic snapshot isolation technique similar to [how transactions work in Starcounter](/2.3.1/guides/transactions), applied on a disk volume level by the operating system. While VSS snapshots the whole volume, it enables extraction of separate files from a snapshot, which is a point-in-time consistent backup of these files.

**Hot backup of database files using VSS**. Native tools to create and access VSS snapshots are available in Windows Server (`diskshadow.exe`), but are not included in other Windows environments. We will use a tool that works effortlessly on all modern Windows operating systems (desktop and server). The below instruction explains how to backup the most recently available VSS snapshot of all files from a given location to another given location.

Objective: backup a location `c:\Users\User\Documents\Starcounter\Personal\Data\Default\Default-20150915T100722555\` to `Y:\Backup`.

Start by downloading the latest version of the `shadowspawn.exe` utility from <https://github.com/candera/shadowspawn/downloads>. Put the utility on your machine in a globally available location, such as `C:\Windows\System32`.

Run this backup command

```
shadowspawn C:\Users\User\Documents\Starcounter\Personal\Data\default\Default-20150915T100722555\ Q:\ robocopy Q:\ "Y:\Backup\%date:~10,4%.%date:~4,2%.%date:~7,2%-%TIME:~0,2%.%TIME:~3,2%.%TIME:~6,2%" /s
```

This will create a shadow copy of a drive, mount shadowed copy of a source folder to `Q:` (the drive letter `Q` must be free, otherwise choose other letter), copy contents of `Q:\` to a sub-folder in `Y:\Backup` named after the current date and time, such as `Y:\Backup\2015-09-15_16-41-48`, unmount and destroy a shadow copy. Copying is done with `robocopy` utility that ships with Windows, you can use any other command, e.g. you can call your favorite incremental binary backup utility and store the result in a cloud.

{% hint style="info" %}
Read more about robocopy in the [robycopy documentation](https://www.computerhope.com/robocopy.htm).
{% endhint %}

**Note:** creating and removing a VSS snapshot during the described backup routine may affect performance of your Starcounter application when you have user activity peaks. Consider running backup scenario in periods of time when you have less than hundreds of thousands of simultaneously connected users.

**Working with VSS snapshots.** VSS snapshot is a powerful tool. You can use native Windows tool `vssadmin` to manage snapshots in a command line. We recommend a GUI-based tool [ShadowExplorer](http://www.shadowexplorer.com/) (free) and [Z-VSSCopy](http://www.z-dbackup.com/vss-shadow-copy-open-file-backup.html) (free for non-commercial use) to create, view and mount VSS snapshots.

**Files to backup:**\
The files necessary for backup are:

* `<database_name>.cfg`
* `<database_name>.*.log`
* `<database_name>.*.optlog`

Follow these steps to find these files:

1. Go to the server repository that is found, by default, at `%UserProfile%\Documents\Starcounter\Personal`. If configurations have been done in the installer, then it might reside somewhere else. In that case, it can be found by following the `<server-dir>` path at `Program Files\Starcounter\configuration\Personal.xml`, if the default setting were accepted in the installer.
2. Open `Personal.server.config` in this repository.
3. Go to the path specified by `<DatabaseDirectory>`. It's normally something like: `%UserProfile%\Documents\Starcounter\Personal\Databases`.
4. In this directory, there will be a directory for each database with the same name as the database.
5. Open the directory for the database that will be backed up.
6. Open the file with the name `<database_name>.db.config`.
7. Find the parameter `TransactionLogDirectory`and go to the path specified. There you will find the files that are necessary for backup.
8. Use the directory above for the VSS backup.

In the case that there is no full `.log` file in the directory, there will be no `.optlog` file.

## Restoring a Database From a Backup

Database is a set of files you have made backup for. To recover your database to one of the backups, you can create a new database in Starcounter Administrator and then, without starting it, copy all backup files into that database data folder. After doing so, you can shut down the old database and run your application on a new database with minimal downtime. If you have a failover machine, you can switch your web-frontend to route requests to a failover machine. The simplest way to restore a database from backup is to stop Starcounter, remove all database data files, copy backup files in data files location and start Starcounter.

## Starcounter in a Virtual Machine

Virtual machine is a convenient tool to make production environment easily deployable and manageable. For Starcounter apps running in virtual machine, the simplest way to do cold backup is to shut down VM, copy VM image, turn on VM. To perform hot backups of VMs, one should use Hyper-V facilities for shadow copying and [Hyper-V asynchronous replication](http://blogs.technet.com/b/yungchou/archive/2013/01/10/hyper-v-replica-explained.aspx) for disaster recovery. Please refer to the [official Microsoft documentation](https://technet.microsoft.com/en-us/library/jj134172.aspx) on Hyper-V high availability features. Note that, since Starcounter is capable of doing millions of DB transactions per second on a single commodity server, it has a nice overcapacity to run with extreme performance even inside a VM.

## SSD or HDD?

Starcounter streams changes to disk sequentially. Modern HDDs can stream terrabytes of data in seconds with sequential writes. That's why you can run your app suite on big commodity HDDs with our in-memory engine safely and performantly. Choice of HDD, SSD or PCI flash appliance will not affect your read performance at all, in most cases it is a choice of your preference. What is more, with cheap HHDs you can setup inexpensive RAID-1 array to make your solution be tolerant towards hardware faults and make your regular backups onto more disks. If you run an application with a vast amount of writes (more that 30% of writes among all database operations), consider going for faster storage devices to increase throughput while having tens of thousands of users operating simultaneously.

## Disabling Anti-Virus Scan of Database Files

For higher performance, disable your anti-virus from scanning the database files in the [server directory](/2.3.1/guides/working-with-starcounter/configuration-structure#server-repository).&#x20;

## Checklist: Starcounter in Production

1. Always have enough spare space for the database log files and images on your hard-drive: 2-3x RAM.
2. Install task to clean `.log.bak` files in Windows Task Scheduler. Avoid disk overflow with `.bak` files that you don't need! Verify that the task is actually executed as expected, since you might miss certain task settings so that Windows disallows it to run.
3. Verify that a VSS-based backup routine is set. Always perform backups on a regular basis. Ensure that backup goes to a separate drive or to several independent drives. Please always remember that if you copy those files to a separate machine and use it as a failover replica, that replica is asynchronous!
4. For hot (seamless downtime) failover: consider synchronous replication (or adjustable asynchronous replication, when applicable in your business domain) with Windows Server 2016 Datacenter Edition Storage Replica and Failover Cluster features, or [3-rd party solutions](http://www.evidian.com/products/high-availability-software-for-application-clustering/file-replication-byte-level-with-failover-mirror-cluster/) available.
5. For hot asyncronous failover: consider running Starcounter is a VM over [Hyper-V hypervisor with Replica feature](http://blogs.technet.com/b/yungchou/archive/2013/01/10/hyper-v-replica-explained.aspx). This also enables a convenient backup scheme: the replica will be the backup of the recent point-in-time for the master machine.
6. For warm (seconds downtime) synchronous failover: consider [shared disk setup](https://en.wikipedia.org/wiki/Shared_disk_architecture) (SAN or NAS + round-robin front-end machine). Assert that there should always be a single master in such scheme: only one machine at a time can be active, several machines must not write to the same data files on a shared disk! As a commodity alternative, Microsoft Windows Server 2008, 2008 R2, 2012 and 2012 R2 provides a new feature called [cluster shared volume](https://en.wikipedia.org/wiki/Cluster_Shared_Volumes). Coupled with Hyper-V Failover Cluster
7. Running on-premise: consider RAID-1 as a failover tool. The failure rates of modern HDDs vary from 1% to 10%, hence running some RAID-based solution is a necessity. In case of a cloud provider, most likely it will take care of making your data secured for hardware failures. However, problems occur in clouds sometimes, so you should care of backing up your data in the cloud environment.

**Please always remember that your data is one of your most important assets!** Implement the proper strategy for backup, failover and disaster recovery from day one. Using advices from the article, you can make your data safety strategy gradual. First of all, start with simple Shadow Copy backups. Use Dropbox for Business and asynchronous cryptography (with private and public keys) to store terabytes of backups for just hundreds of dollars per year. If you run Starcounter in a cloud like Amazon, then you most likely run Windows Server 2012 R2 Standard, which ships with Failover Clustering role out of the box. Use the facilities like Failover Clustering that go for free with your operating system. Create a simple Failover Cluster with one master machine and one hot-standby machine which would always be a replica of your master machine. Store your database image files on a Cluster Shared Volume, so that the hot-standby machine will continue to run on exact the same data. It will take some seconds for the replica machine to start Starcounter (since two machines cannot open the same database at one time), so the resulted failover will be warm. However, today you can achieve true synchronous hot failover using Windows Server 2016 Storage Replica (Datacenter edition is required).

![](https://3760276903-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2Fstarcounter%2Fe48b74d0-8680-11e7-9944-1f85270462c6%2Fe4c7e190-8680-11e7-9944-1f85270462c6%2FScreen-Shot-2015-10-06-at-11.24.041-1024x726.png?generation=1503327412270467\&alt=media)




---

[Next Page](/llms-full.txt/1)