Features and Targets in Machine Learning: What the Model Can Actually See

Every supervised learning problem is a question about a table. The hard part is knowing which columns the model is allowed to look at.

The spreadsheet question every beginner gets wrong

You have a dataset. It is a spreadsheet with rows and columns, and somewhere in that grid is the answer to a prediction problem. Maybe you want to forecast house prices, predict whether a customer will churn, or estimate delivery times. You open the file, and the columns stare back at you like a crowd of strangers.

Which ones feed the model? Which one is the answer? And why do some columns feel useful but should never touch your model?

Beginners get this wrong all the time, and the mistake is not about math. It is about a missing mental model. Once you see the structure clearly, most of the confusion disappears.

Here is the organizing question for everything in this article: What can the model actually see when it needs to make a prediction?

Every column in your dataset is either the answer you want, a candidate input that might help you predict it, or something that must be excluded. The trick is telling the three apart.

Rows are examples, columns are features

Imagine you are predicting house prices. Your dataset might look like this:

Each row is one example — one house, one observation, one instance. Each column is a measurable characteristic of that house: its size, its number of bedrooms, its neighborhood quality.

A feature is an individual measurable characteristic or property of the thing you are observing. In a table, features live in columns. The model learns patterns across rows: houses with more square footage tend to sell for more, and location matters too.

The block of input columns — everything the model reads to make its prediction — is called the feature matrix. Think of it as the table with the answer column removed. The model never sees the answer during prediction. It only sees the features, and it uses what it learned during training to guess the answer for new rows.

A feature is not magic. It is just a piece of information about an example that might help you predict the outcome. The question is always whether that information would exist at the moment you need the prediction.

The target is the answer you want to predict

The target is the outcome column — the answer the model is trained to predict. In the house-price example, the target is price. Everything else is a candidate feature that might help explain it.

In supervised learning, the training data pairs features with known target values. The model studies many examples of "these inputs led to this outcome," and it learns the mapping between them. Later, when you hand it a new row with no price attached, it uses that learned mapping to make its best guess.

Targets come in two common flavors:

• Categorical targets — the answer is one of a few categories. Spam or not spam. Will churn or stay. This is called classification.
• Continuous targets — the answer is a number on a scale. Price, temperature, delivery time. This is called regression.

You may have met this idea in a statistics class under different names. Statisticians often call the target the dependent variable and the features the independent variables. Same structure, different vocabulary. The target depends on the features, at least in the sense that the model tries to explain the target using them.

One important detail: in many datasets, the target column is also called the label. The label is the known, correct answer attached to each training example. When people talk about "labeled data," they mean rows that come with their answers attached.

Identifiers: a warning label, not an automatic verdict

Now we get to the trap.

Real datasets almost never arrive with only features and a target. They arrive with extra columns that look informative but need careful judgment. The most common are identifiers.

An identifier is a column that names a row or an entity. Customer ID. Email address. Transaction code. Row number. These columns deserve suspicion, but they are not all the same.

Consider two very different identifier-shaped columns:

• Row number is pure record-keeping. It was assigned by whoever exported the spreadsheet. It describes nothing about the house, the customer, or the event. It cannot help predict anything about a new row, because a new row simply gets the next number.
• Customer ID is trickier. The ID itself is arbitrary, but it points to a customer with history. If you join that ID to a table of past purchases, the ID becomes a way to say "this customer has bought from us three times before." That repeat-purchase history is a real, learnable signal.

So the honest rule is: an identifier is a warning label, not an automatic verdict. Ask what the identifier points to and whether that information can generalize to the cases you will predict.

Why does this matter? Two failure modes hide in identifier-shaped columns.

First, a model can memorize a unique value. If every row has its own ID, the model can learn "ID 4821 leads to a high price" without learning anything about what makes a house valuable. It looks great on the training data and fails completely on new rows, because new rows have IDs the model has never seen.

Second, a model can memorize the entity behind the ID. If your dataset contains many rows for the same customer, a model can learn "customer 77 always buys" — which is only useful if customer 77 is exactly the person you will predict next. For a new customer, that memorization is worthless.

The prediction-time test catches the first problem immediately: when you need a real prediction for a new house, will its row number exist? No. The row number is created for record-keeping, not for describing the house. It does not belong in the features.

But the customer ID case needs a second question: does this value describe a property that transfers to the cases I will actually predict? If the answer is no, exclude it. If the answer is yes — because the ID connects to history you can also retrieve at prediction time — then the history is the feature, not the raw ID.

Common mistake: Dropping every column that looks like an ID without asking what it represents. A random row number is useless. A customer ID that links to purchase history may point at your most valuable signal. The column name is not the verdict; the meaning is.

Leakage: when the answer sneaks into the inputs

Identifiers are the obvious trap. Leakage is the subtle one.

Leakage happens when a feature contains information about the target that would not actually be available at prediction time. The model learns a shortcut instead of a real relationship, looks brilliant during training, and collapses in the real world.

Here is a classic example. Suppose you want to predict whether a customer will buy a product. Your dataset includes a column called SpokeToCustomerAgent, and it is set to True for customers who talked to an agent. The model quickly learns: SpokeToCustomerAgent = True means the customer bought the product.

But here is the catch: the customer agent is only assigned after the customer has already purchased. At the moment you need a prediction — before the purchase happens — that column would be empty. The feature is a proxy for the answer. The model did not learn why people buy. It learned to read the ending of the story.

Leakage can also come from future-dated columns, like a "total spent" field that is only filled in after the billing cycle closes, or a "resolved" flag that appears only after a support ticket is closed.

The consequence is the same: a model that performs beautifully in training and fails the moment it meets real data. The prediction-time test catches leakage too. If a column's value would not exist when you need the prediction, it does not belong in the features.

Note: Leakage is often called label leakage because the feature is secretly carrying information about the label — the target — into the inputs.

A quick way to sort any dataset's columns

You do not need to be an expert to classify columns. You need three questions, asked in order:

1. Is this the outcome I want to predict? Then it is the target. Remove it from the features.
2. Would this value exist at the moment I need the prediction? If no, exclude it. This catches leakage and pure record-keeping columns.
3. Does this value describe a property that transfers to new cases? If it is just a unique name or row key, exclude it. If it points to real history or a meaningful group, consider using the meaningful part as a feature.

The same table can be read differently depending on the question you are asking. In a house-price dataset, price is the target and square footage is a feature. But if you were building a model to predict square footage from price and location, the roles would flip. A column is not permanently a feature or a target. Its role depends on the prediction problem you are solving.

What matters is the honest framing: what information exists at the moment you need the answer, and does that information describe something that will also exist for the cases you predict next? Features are what the model can see. The target is what you want it to figure out.

The durable takeaway

Every supervised learning problem is a question about a table. The honest way to frame that question is to ask what the model can actually see at prediction time.

Features are the inputs — the measurable characteristics that describe each example. The target is the answer the model is trained to predict. And the columns in between demand judgment: identifiers are suspects to interrogate, not criminals to sentence on sight, and leakage is the answer sneaking into the inputs wearing a disguise.

The next time you open a dataset, do not stare at the columns hoping for clarity. Sort them. Ask the three questions. Build your feature matrix deliberately, keep the target separate, and cut everything that would not exist in the real moment of prediction.

From there, the real work of data preparation begins: cleaning messy values, handling missing data, and turning raw rows into a defensible feature matrix. But that work only matters if you have the right columns in the first place. Get the framing right, and every later step gets easier.