•Observation :
Examples using public third-party APIs. Not affiliated with or endorsed by any API provider. All trademarks belong to their respective owners.
I'm Pires Aurélio , a Go/Python developer from Angola. I build scrapers in production and document everything in public.
My work:
For 2 years I scraped sites with pure Python/Go. Then I hit Google. Everything broke.
This tutorial is how I solved it with Go + SerpApi.
1. Why My Python Scraper Failed on Google
My repo py-websites-scraper uses asyncio + aiohttp + readability-lxml. Great for blogs.
But for Google Jobs / Shopping:
- CAPTCHA in 10 requests
- 429 blocks
- Selectors change weekly
I was spending 80% time fighting anti-bot, 20% building features. Wrong.
2. Why SerpApi + Go Is The Stack
SerpApi handles proxies, CAPTCHAs, parsing. You get clean JSON.
Go handles concurrency perfectly.
My stack now: Go + SerpApi + Gin REST API (as I teach in my blog)
3. Setup (2 minutes)
go mod init go-serpapi-demo
go get github.com/serpapi/google-search-results-golang
go get github.com/gin-gonic/gin
export SERPAPI_KEY=your_free_key_from_serpapi.com
#. Example 1: Google Jobs - (Fixed):
package main
import (
"fmt"
"log"
"os"
g "github.com/serpapi/google-search-results-golang"
)
func main() {
apiKey := os.Getenv("SERPAPI_KEY")
if apiKey == "" {
log.Fatal("SERPAPI_KEY is not set")
}
params := map[string]string{
"engine": "google_jobs",
"q": "Go Developer",
"location": "Remote",
"hl": "en",
}
search := g.NewGoogleSearch(params, apiKey)
results, err := search.GetJSON()
if err!= nil {
log.Fatalf("SerpApi request failed: %v", err)
}
jobsRaw, ok := results["jobs_results"].([]interface{})
if!ok {
log.Printf("No jobs_results in response: %+v", results)
return
}
if len(jobsRaw) == 0 {
fmt.Println("No jobs found")
return
}
for _, j := range jobsRaw {
jobMap, ok := j.(map[string]interface{})
if!ok {
continue
}
fmt.Printf("[%v] %v - %v\n", jobMap["via"], jobMap["title"], jobMap["company_name"])
}
}
#. Example 2: Google Shopping Price Tracker: ( Fixed):
package main
import (
"fmt"
"log"
"os"
g "github.com/serpapi/google-search-results-golang"
)
func main() {
apiKey := os.Getenv("SERPAPI_KEY")
if apiKey == "" {
log.Fatal("SERPAPI_KEY is not set")
}
params := map[string]string{
"engine": "google_shopping",
"q": "golang book",
}
search := g.NewGoogleSearch(params, apiKey)
results, err := search.GetJSON()
if err!= nil {
log.Fatalf("failed to get shopping results: %v", err)
}
shoppingResults, ok := results["shopping_results"].([]interface{})
if!ok {
log.Printf("No shopping_results found: %+v", results)
return
}
fmt.Printf("Found %d products\n", len(shoppingResults))
for _, item := range shoppingResults {
if m, ok := item.(map[string]interface{}); ok {
fmt.Printf("- %v: %v\n", m["title"], m["price"])
}
}
}
______________
By:Pires Aurélio
Contact: *Emails are not allowed*
https://blogspotangola.blogspot.com/?m=1
https://www.linkedin.com/in/pires-aur%C3%A9lio-511330385