Lab - Extend the Schema and Query

In this lab you will extend the product schema from Lab 1 with more fields, feed a richer dataset, practice partial updates, and write YQL queries with filters, sorting, and pagination. You will also add a custom document summary.

Prerequisites

Your Vespa container from Lab 1 should be running with the ecommerce-app deployed. If you stopped it, restart with docker start vespa.

Extend the Schema

Open ecommerce-app/schemas/product.sd and replace its contents with this expanded schema:

schema product {
  document product {
    field title type string {
      indexing: summary | index
      index: enable-bm25
    }

    field description type string {
      indexing: summary | index
      index: enable-bm25
    }

    field category type string {
      indexing: summary | attribute
      attribute: fast-search
    }

    field brand type string {
      indexing: summary | attribute
      attribute: fast-search
    }

    field price type int {
      indexing: summary | attribute
    }

    field rating type float {
      indexing: summary | attribute
    }

    field in_stock type bool {
      indexing: summary | attribute
    }

    field color type string {
      indexing: summary | attribute
    }

    field updated_at type long {
      indexing: summary | attribute
    }
  }

  fieldset default {
    fields: title, description
  }

  document-summary short {
    summary title {}
    summary price {}
    summary brand {}
    summary in_stock {}
  }
}

What changed from Lab 1:

• index: enable-bm25 on title and description enables BM25 scoring (we will use this in Lab 4)
• description is a new indexed text field for full-text search
• brand and category have attribute: fast-search for efficient filtering
• rating, in_stock, color, and updated_at are new attribute fields
• A default fieldset lets you search title and description together
• A short document summary returns only a few fields, reducing response size

Redeploy

vespa deploy --wait 300 ecommerce-app

Vespa applies schema changes live. Your existing data is still there, but the old 10 documents do not have the new fields. Let's re-feed with a richer dataset.

Feed the Extended Dataset

Create feed-extended.jsonl. Here is a larger set of products with all fields populated:

{"put": "id:product:product::1", "fields": {"title": "Cotton crew neck t-shirt", "description": "Soft breathable cotton t-shirt with a classic crew neck. Perfect for everyday wear in warm weather.", "category": "Tops", "brand": "BasicWear", "price": 29, "rating": 4.2, "in_stock": true, "color": "White", "updated_at": 1709251200}}
{"put": "id:product:product::2", "fields": {"title": "Slim fit jeans dark blue", "description": "Classic slim fit jeans in dark blue wash. Stretch denim for comfort throughout the day.", "category": "Bottoms", "brand": "DenimCo", "price": 79, "rating": 4.5, "in_stock": true, "color": "Blue", "updated_at": 1709164800}}
{"put": "id:product:product::3", "fields": {"title": "Running shoes lightweight mesh", "description": "Lightweight running shoes with breathable mesh upper and responsive foam sole. Great for daily training.", "category": "Shoes", "brand": "SpeedStep", "price": 120, "rating": 4.7, "in_stock": true, "color": "Black", "updated_at": 1709078400}}
{"put": "id:product:product::4", "fields": {"title": "Waterproof hiking jacket", "description": "Fully waterproof jacket with sealed seams and adjustable hood. Three-layer fabric for mountain conditions.", "category": "Outerwear", "brand": "TrailPeak", "price": 189, "rating": 4.8, "in_stock": false, "color": "Green", "updated_at": 1708992000}}
{"put": "id:product:product::5", "fields": {"title": "Leather crossbody bag", "description": "Genuine leather crossbody bag with adjustable strap. Multiple compartments for phone, wallet, and essentials.", "category": "Accessories", "brand": "UrbanCraft", "price": 65, "rating": 4.1, "in_stock": true, "color": "Brown", "updated_at": 1708905600}}
{"put": "id:product:product::6", "fields": {"title": "Wool blend overcoat", "description": "Elegant wool blend overcoat with notch lapels. Fully lined for warmth in cold weather.", "category": "Outerwear", "brand": "ClassicLine", "price": 250, "rating": 4.6, "in_stock": true, "color": "Grey", "updated_at": 1708819200}}
{"put": "id:product:product::7", "fields": {"title": "Striped cotton polo shirt", "description": "Classic polo shirt with horizontal stripes. Ribbed collar and two-button placket.", "category": "Tops", "brand": "BasicWear", "price": 45, "rating": 4.0, "in_stock": true, "color": "Blue", "updated_at": 1708732800}}
{"put": "id:product:product::8", "fields": {"title": "High-waist yoga leggings", "description": "Four-way stretch yoga leggings with high waistband and hidden pocket. Moisture-wicking fabric.", "category": "Bottoms", "brand": "FlexFit", "price": 55, "rating": 4.4, "in_stock": true, "color": "Black", "updated_at": 1708646400}}
{"put": "id:product:product::9", "fields": {"title": "Canvas sneakers white", "description": "Classic white canvas sneakers with vulcanized rubber sole. Minimal design goes with everything.", "category": "Shoes", "brand": "StreetWalk", "price": 60, "rating": 4.3, "in_stock": true, "color": "White", "updated_at": 1708560000}}
{"put": "id:product:product::10", "fields": {"title": "Stainless steel watch", "description": "Stainless steel watch with sapphire crystal glass and Japanese quartz movement. Water resistant to 50m.", "category": "Accessories", "brand": "TimeCraft", "price": 199, "rating": 4.9, "in_stock": true, "color": "Silver", "updated_at": 1708473600}}
{"put": "id:product:product::11", "fields": {"title": "Fleece zip hoodie", "description": "Warm fleece hoodie with full zip front and kangaroo pockets. Soft brushed interior.", "category": "Tops", "brand": "BasicWear", "price": 59, "rating": 4.3, "in_stock": true, "color": "Grey", "updated_at": 1708387200}}
{"put": "id:product:product::12", "fields": {"title": "Linen shorts casual", "description": "Relaxed fit linen shorts with drawstring waist. Lightweight and breathable for summer.", "category": "Bottoms", "brand": "SummerEase", "price": 42, "rating": 3.9, "in_stock": false, "color": "Beige", "updated_at": 1708300800}}
{"put": "id:product:product::13", "fields": {"title": "Leather ankle boots", "description": "Full grain leather ankle boots with side zip. Durable rubber outsole and cushioned insole.", "category": "Shoes", "brand": "TrailPeak", "price": 165, "rating": 4.6, "in_stock": true, "color": "Brown", "updated_at": 1708214400}}
{"put": "id:product:product::14", "fields": {"title": "Packable rain jacket", "description": "Ultra-light rain jacket that packs into its own pocket. Waterproof with taped seams.", "category": "Outerwear", "brand": "TrailPeak", "price": 85, "rating": 4.2, "in_stock": true, "color": "Navy", "updated_at": 1708128000}}
{"put": "id:product:product::15", "fields": {"title": "Silk scarf printed", "description": "Luxurious silk scarf with hand-rolled edges. Bold geometric print in multiple colors.", "category": "Accessories", "brand": "ClassicLine", "price": 78, "rating": 4.5, "in_stock": true, "color": "Red", "updated_at": 1708041600}}
{"put": "id:product:product::16", "fields": {"title": "Denim jacket classic", "description": "Classic denim jacket in medium wash. Button front with chest pockets and side pockets.", "category": "Outerwear", "brand": "DenimCo", "price": 95, "rating": 4.4, "in_stock": true, "color": "Blue", "updated_at": 1707955200}}
{"put": "id:product:product::17", "fields": {"title": "Performance tank top", "description": "Lightweight tank top with mesh back panel for ventilation. Quick-dry fabric for intense workouts.", "category": "Tops", "brand": "FlexFit", "price": 32, "rating": 4.1, "in_stock": true, "color": "Black", "updated_at": 1707868800}}
{"put": "id:product:product::18", "fields": {"title": "Chino pants slim fit", "description": "Slim fit chino pants in stretch cotton twill. Versatile enough for office or weekend.", "category": "Bottoms", "brand": "ClassicLine", "price": 68, "rating": 4.3, "in_stock": true, "color": "Khaki", "updated_at": 1707782400}}
{"put": "id:product:product::19", "fields": {"title": "Trail running shoes", "description": "Rugged trail running shoes with aggressive tread pattern. Rock plate protects against sharp terrain.", "category": "Shoes", "brand": "SpeedStep", "price": 140, "rating": 4.7, "in_stock": false, "color": "Orange", "updated_at": 1707696000}}
{"put": "id:product:product::20", "fields": {"title": "Polarized sunglasses aviator", "description": "Aviator style sunglasses with polarized lenses and metal frame. UV400 protection.", "category": "Accessories", "brand": "UrbanCraft", "price": 45, "rating": 4.0, "in_stock": true, "color": "Gold", "updated_at": 1707609600}}

Feed it:

vespa feed feed-extended.jsonl

Since we are using the same document IDs for products 1-10, Vespa overwrites the old documents with the new ones that include all fields. Products 11-20 are new.

Practice Partial Updates

Partial updates let you change individual fields without re-feeding the entire document. Let's mark the hiking jacket as back in stock and update the trail running shoes price:

vespa document update id:product:product::4 '{"fields": {"in_stock": {"assign": true}}}'

vespa document update id:product:product::19 '{"fields": {"price": {"assign": 129}}}'

Verify the updates worked:

vespa document get id:product:product::4

The in_stock field should now be true.

Write Queries

Text search across the fieldset

Search both title and description at once using the default fieldset:

vespa query "select * from product where default contains 'leather'"

This matches the crossbody bag (title), the ankle boots (title and description), and potentially others with "leather" in the description.

Filter by attribute

Find all products from the TrailPeak brand:

vespa query "select * from product where brand = 'TrailPeak'"

Combine text search and filters

Search for shoes under $130:

vespa query "select * from product where default contains 'shoes' and price < 130"

Sort results

Get all products sorted by price, cheapest first:

vespa query "select * from product where true order by price asc"

Sort by rating, highest first:

vespa query "select * from product where true order by rating desc"

Pagination

Get the first page of 5 results, then the second page:

vespa query "select * from product where true order by price asc" "hits=5" "offset=0"

vespa query "select * from product where true order by price asc" "hits=5" "offset=5"

Use the short summary

Request just the fields defined in the short document summary:

vespa query "select * from product where true" "summary=short" "hits=5"

The response includes only title, price, brand, and in_stock for each hit, which is less data to transfer.

Find out-of-stock products

vespa query "select * from product where in_stock = false"

Checkpoint

Run this query and verify the result:

vespa query "select * from product where brand = 'BasicWear' and price < 50" "summary=short"

You should see 2 results: the cotton t-shirt (29) and the polo shirt (45). The response should only contain the short summary fields.

What You Built

Your application now has:

• A schema with 9 fields covering text, numeric, boolean, and timestamp data
• enable-bm25 on text fields for proper relevance scoring (used in Lab 4)
• fast-search on category and brand for efficient filtering
• A default fieldset for searching title and description together
• A short document summary for lightweight responses
• 20 products with realistic data
• Experience with partial updates, filtering, sorting, and pagination

In the next lab, you will structure this into a proper application package with query profiles.