dbt Core Tutorial 3 - Committing Changes

Hello. It's Jason from Can here. And in this video, I'd like to discuss how to use our new GitHub integration to make changes to your dbt project directly from the canvas. So in this canvas, we've got a single database connected, a BigQuery one, and it's got some dbt connection as well. Now in this case, we're connected to GitHub, so we're looking at the demo branch. And first things first, let's look at how to add a new model to this branch. So to do this in count, we just create a new cell. We fill out some contents of the model. It's not a very exciting one. And we give it maybe perhaps a better name. So at this point, this is just a regular cell. It's got nothing to do with dbt. So to tell Kent that we want this to be considered as a model, well then there's a toggle switch over here. If you enable this, you'll notice there's a dbt icon that appears in the corner. And if we go to the new commit to GitHub button, we'll see that in our list of changes, we are going to add a new file. In this case, we're adding a file called mymodel. Sql. This is just the name of the cell. And we're placing it in the default models directory that's in the dbt settings account. We've also got a default commit message. We could open a pull request if we wanted to with a default title and a default body. But we won't do that just yet. If we wanted to not just add but delete a model, then we might want to say delete this failing test. So if we add this to the canvas, we don't need to set this to be a dbt model because we've already pulled it in from the sidebar. We can't know that that's a model already. And to delete this model, we just click the delete button. Now nothing has actually been deleted yet. All we've done is clear the contents of the cell. That's a signal to count that we want this model to be deleted. So if we head back to our view of diffs, then you can see there's now a second diff where this d icon means that we should be deleting this file. To edit models, very similarly, we just find the model we want to edit in our sidebar, add it to the canvas just by itself. So this is just the plain model text as we expect. And in this case, let's just say re end the column. If we head back to our diff view again, you'll see there's a third diff now where we've taken the original macro text or model text from dbt And we've updated this column reference. So we could commit these changes in one go if we wanted to. But there's actually one more thing to discuss when it comes to committing changes to models that's important to understand. So at the moment, we've just been importing text directly from GitHub that's compatible with dbt and tweaking it in cells. But there's another kind of more powerful way to do this in count, so let's investigate how that works. Let's create a model by, I don't know, referring to an existing model. We'll call this new model. So at the moment, this isn't considered a model. So let's do that. But, also, one thing I'll notice is that this isn't valid dbt syntax. If we sent this to dbt, dbt would have no idea what to do with it. Like, what is macros? It's a selling count. It's got nothing to do with dbt. So to investigate how count makes this possible, we can preview what's going on using the copier SQL model. So the default view here is just the full compiled SQL. So this is SQL that is okay to be shipped to a database to be executed. This is exactly what count sends to a database to execute this cell. So we've taken this upstream cell macros. We've compiled it in as plain SQL and inserted it as a CTE. And you can see here, this simple tab on the left shows you what's going on. We're trying to compile new model, and we need macros to do it. What's useful for dbt is the SQL and Jinja option. So this changes the compilation step to not look at pure compiled SQL, but to include Jinja as well. So can't use to make some changes here to get this to work with dbt. And you can see what's going on by looking at this code changes description on the left. And because we know that this new model should be considered a candidate as a dbt model, it cannot refer to other cells using plain SQL references. It needs to use a ref. So in this case, we've modified the text of this cell to take this macro SQL reference and add a ref alias. Now it might be the case that you're not referring to a model. You're referring to another plain count cell. So to prove you what this looks like, you can actually deselect this dbt icon over here. And this tells count to consider macros as not a dbt model, but just a plain regular cell. In this case, it changes the model that we compile. So we insert the model definition as a CTE, but with Jinja intact. So we haven't compiled this out to SQL. And because we've done this, actually, we don't need to take a ref reference down here. This can just be a plain SQL reference because we've added the definition up here. This can get a bit complicated. So to check what's going on, you can always read what's happening down in the corner here. But this process is what happens under the hood when we make commits for GitHub. So if I do just what I described before, deselect the model at the top, select model down here, go to commit to GitHub, and you'll see that this new model that we create is exactly what we saw in that preview. We're injecting the ginger text from the macros model as a CTE. And now this is a model. Sql file that can be committed to GitHub and can be safely consumed by dbt. So that's a quick overview of how the model compilation and model export process goes in Kapt. If you have any more questions, you can read more in our documentation, or you can reach out to us using the in app chat or in the Slack community.