Building a Catalog

In this tutorial, we're going to start from scratch and walk through how to set up your very first count metrics catalog. By the end, you'll be ready to unlock faster, clearer, and more consistent analytics for your team. Let's jump in. First, head over to your account workspace. On the left hand side, you'll see a section called catalogs. This is where all your catalogs live. Click on create new catalog to add a new catalog. This will create a new space to define the metrics and SQL logic you'll reuse later. Give your catalog a clear descriptive name. For this example let's call it Spotify Data Insights. Good names matter. It helps your teammates instantly know what the catalog is for. Next, let's add a description. This is completely optional. It's worth noting that all of the properties we're defining here can be edited at any point. So if you're stuck for ideas, don't sweat it. We can change it later. Next, let's add a data source. You'll be prompted to choose from the list of connections available in your workspace. A connection is available if first it exists in your workspace and second you have the right access to it. If you don't see your connection in the list, it could mean the connection hasn't been set up yet, but it could also mean that you don't have permission to view the connection. Sometimes you may see the connection, but it's grayed out. That means the connection exists and you do have the access to view it, but you don't have the correct admin access to add it to a catalog. So to bring a data source into the catalog, two things are required. The connection must be set up in your workspace, and you must have admin access to it. In this example, I only have one connection that meets both requirements, which is fine by me because this is the one I want. To add the connection to your catalog, click the toggle next to it and then click next. Once you've added your data connections, you can have one or more linked to your catalog. You'll see a list of tables you can pick from to auto generate views. This is my favorite way to create views because it's quick and requires almost no input. All you need to do is select the objects you want to appear in your catalog from the list provided. I recommend this if you want one to one views. For example, if your source data platform already has a semantic layer you can simply mirror those objects. It also works well if you want to make small modifications because it gives you a great starting point to build from. If your schema has just a handful of objects, scrolling is fine, but if you have dozens or even hundreds, the search function becomes really useful. Let's use this now to narrow down the list. I'll bring in just a couple of these for this example. Next you're going to bring in additional members. If you're building your catalog collaboratively, which I recommend, the earlier you add your co collaborators the smoother the experience will be. Since this is just a tutorial I won't be adding members right now but this is the dialogue where you would do it. To add a member simply search their name and select them from the workspace member drop down. From there you can assign a role to them. We have documentation that explains the available catalog roles and you'll also see some details about them right here in the UI which I'll show you in a moment. We've done most of the heavy lifting on setup and now we're in the catalog YAML editor. In the center of the screen you'll see the YAML code. These initial views and datasets are auto generated and you'll notice the comment at the top marking them as such. The header comment also contains the fully qualified table name of the object that the view was generated on. We've also set up default caching. So by default we set up caching with a duration of three thousand six hundred seconds or one hour. You can remove this setting without breaking anything by simply deleting these two lines and we'll cover caching in more detail in a future video. Let's have a look at fields. So for each field you'll have a name and this is by default the same as the name of the field on the table that we've generated this view on. If metadata is available on the connection we'll bring in custom labels and descriptions. If we don't have metadata containing descriptions or custom labels then we will generate user friendly labels by default. So for example, by removing underscores and capitalizing the first word. So take a look at album ID for this one. For numeric non primary key fields, we also generate a default list of aggregates. In the UI these appear as an expandable drop down so end users can select pre aggregated values. The same applies to date fields. We automatically generate a default set of time frames which appear in an expandable drop down for easy roll up selection. Take a look at release date here. If your source table has a primary key we'll bring that through as well. You can see that here in this example. By default you'll get one data set per view. Let's take a look at the data set. To publish these views all I need to do is click validate and commit. This opens the validation checker. Once you're in production it will compare your changes against every canvas using the catalog and let you know if the change breaks something, fixes something, or has no effect. Based on that feedback you can decide whether to publish. In this example, I don't have any projects connected to this catalog yet so I'm free to commit my changes. Let's go to the catalog home page. Think of the catalog home page as the dashboard where you can edit or simply view your catalog. On screen you'll see sections for views and datasets. Each rectangle represents a single object. Clicking into a view shows details about its fields including their label this is what your users will see in the canvas, name this is the system name that is used behind the scenes, their data type and any description that you have set up for your view. The references tab will also show the canvases and users that are relying on that view so you can track how your data is being consumed. Datasets are very similar except they contain information about views instead of fields. You can see the view display name that your end users will be able to see, the system name of the views, and if you have a description set up you will see the description here. If you want to go back into the YAML editor you can click edit catalog to add new views or update existing ones. On the right side of the home page you can see a summary here of some catalog properties. How many views, datasets and canvases are in the catalog. You can see when it was created and you can see whether it's connected to any projects. A catalog that is not connected to any projects has not been exposed data wise to any end users. From manage catalog, you can update your catalog name, you can disable sharing, or you can create count secrets for use by your Python cells. You can also delete your catalog which is safeguarded, so to actually delete it you will have to type in the full name of the catalog. You can also add or edit the description from the home page here. So let's go ahead and do that. Head over to manage access to view the users on your catalog. So here you can view who is a member of your catalog and what level of access they have. You can also update their role to either give them more access or take a level away. You can remove members from the catalog, and you can add new ones. You might remember earlier on in the video, I mentioned that there was a place in the UI where there was more information on catalog roles, and that place is here. If you want to add a new data connection or remove one, then the place you need to go is manage data. From manage data, you'll get that same dialogue that we had when we were setting up the catalog, and you're able to switch these toggles any way you need to. I don't need to make any changes here so I won't. And then finally if you want to connect your catalog out to a project so the end users can come in and and really access the data, you can use this manage projects. So clicking on this will give you that list of all of the projects you have access to. Manage projects will show you all of the projects that you have the right accesses to add data sources to, so you must be an admin on that project. As you can see here, I have a list of projects. Three of them are grayed out. This is because while I have access to view those projects and maybe even to create canvases in those projects, I'm not a project admin, so I don't have the correct role to add new data sources to the project. I am a project admin on our customer success training lab, so I can definitely include that there. Be aware that toggling on a project like this makes the data that is in your catalog available to all of the end users who are members of that project. So make sure that you're happy with the catalog before you do this step. The final thing I wanted to show you was how to create a view using one of the catalog canvases. So to do that, click on new canvas. The catalog canvases work a lot like project canvases. You can write SQL, build visualizations, whiteboard, and comment collaboratively. The key difference is that catalog canvases aren't meant to be production dashboards so there's no present mode. Let's create a simple SQL cell. To generate a view from this SQL cell click on the cell and then down the right here you'll notice this export view button. Click on that. Now we can see that this YAML has been created for us and we're back in the catalog YAML editor. In views generated from canvases you'll notice this URL attribute containing a link to the canvas that it was generated from. You'll notice that we don't have any default additions like the aggregates or the labels or the descriptions. These are things that we're going to have to add in ourselves. My new view is already visible in the change log as a new object so all I have to do is go ahead and commit it. Back on the home page you can see the new view here in the middle of the views section. Scrolling down you can see the canvas that we just created it from and then on the right you can see the updated summary metrics showing that there are now three views in this catalogue, one dataset and the canvas. I can see it's connected to one project and that's it. You've just built your very first Calmetrics catalog. From here you can keep adding more datasets, views, and metrics to cover different parts of your business logic.