Python: Script Sliocht Treochtaí Autosuggest Google de Do Eochairfhocail Cuardaigh Nideoige

Is maith le gach duine Google Trends, ach tá sé rud beag fánach maidir le Keywords Long Tail. Is maith linn go léir an t-oifigeach seirbhís treochtaí google chun léargas a fháil ar an iompar cuardaigh. Mar sin féin, cuireann dhá rud cosc ​​ar go leor é a úsáid le haghaidh obair dhaingean;

1. Nuair is gá duit a fháil eochairfhocail nideoige nua, ansin nach leor sonraí ar Google Trends 
2. Easpa API oifigiúil chun iarratais a dhéanamh ar threochtaí google: Nuair a bhainimid úsáid as modúil mar pytrends, ansin ní mór dúinn freastalaithe seachfhreastalaí a úsáid, nó cuirtear bac orainn. 

San Airteagal seo, roinnfidh mé Python Script a scríobh muid chun eochairfhocail treochta a onnmhairiú trí Google Autosuggest.

Torthaí Autosuggest Fetch and Store Le himeacht ama 

Cuir i gcás go bhfuil 1,000 eochairfhocal Síl againn le seoladh chuig Google Autosuggest. Mar chúiteamh, is dócha go bhfaighfimid timpeall 200,000 eireaball fada eochairfhocail. Ansin, caithfimid an rud céanna a dhéanamh seachtain ina dhiaidh sin agus na tacair sonraí seo a chur i gcomparáid le dhá cheist a fhreagairt:

• Cé na ceisteanna eochairfhocail nua i gcomparáid leis an uair dheireanach? Is dócha gurb é seo an cás a theastaíonn uainn. Ceapann Google go bhfuil na ceisteanna sin ag éirí níos suntasaí - trí sin a dhéanamh, is féidir linn ár réiteach Google Autosuggest féin a chruthú! 
• Cé na ceisteanna eochairfhocail a thuilleadh ag claonadh?

Tá an script éasca go leor, agus an chuid is mó den chód a roinn mé anseo. Sábhálann an cód nuashonraithe na sonraí ó rith roimhe seo agus na moltaí a chur i gcomparáid le himeacht ama. Sheachain muid bunachair sonraí comhadbhunaithe mar SQLite chun é a dhéanamh simplí - mar sin tá an stóráil sonraí go léir ag úsáid comhaid CSV thíos. Ligeann sé seo duit an comhad a iompórtáil in Excel agus treochtaí eochairfhocail nideoige do do ghnó a iniúchadh.

Chun an Script Python seo a Úsáid

1. Cuir isteach do shraith eochairfhocal síl ba chóir a sheoladh chuig an autocomplete: eochairfhocail.csv
2. Coigeartaigh na socruithe Script do do riachtanas:
   • TEANGA: “ga” réamhshocraithe
   • TÍR: réamhshocrú “linn”
3. Déan an script a sceidealú le rith uair sa tseachtain. Is féidir leat é a rith de láimh freisin mar ba mhaith leat.
4. Úsáid eochairfhocal_suggestions.csv le haghaidh tuilleadh anailíse:
   • chéad_seen: is é seo an dáta ar tháinig an fiosrúchán den chéad uair san autosuggest
   • feicthe_deireadh: an dáta a bhfaca an fiosrú an uair dheireanach
   • is_nua: if first_seen == last_seen shocraíomar é seo True - Ní gá ach an luach seo a scagadh chun na cuardaigh treochta nua a fháil san autosuggest Google.

Seo Cód Python

# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time

charList = " " + string.ascii_lowercase + string.digits

def makeGoogleRequest(query):
    # If you make requests too quickly, you may be blocked by google 
    time.sleep(WAIT_TIME)
    URL="http://suggestqueries.google.com/complete/search"
    PARAMS = {"client":"opera",
            "hl":LANGUAGE,
            "q":query,
            "gl":COUNTRY}
    response = requests.get(URL, params=PARAMS)
    if response.status_code == 200:
        try:
            suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
        except:
            suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
        return suggestedSearches
    else:
        return "ERR"

def getGoogleSuggests(keyword):
    # err_count1 = 0
    queryList = [keyword + " " + char for char in charList]
    suggestions = []
    for query in queryList:
        suggestion = makeGoogleRequest(query)
        if suggestion != 'ERR':
            suggestions.append(suggestion)

    # Remove empty suggestions
    suggestions = set(itertools.chain(*suggestions))
    if "" in suggestions:
        suggestions.remove("")
    return suggestions

def autocomplete(csv_fileName):
    dateTimeObj = datetime.now().date()
    #read your csv file that contain keywords that you want to send to google autocomplete
    df = pd.read_csv(csv_fileName)
    keywords = df.iloc[:,0].tolist()
    resultList = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}

        for future in concurrent.futures.as_completed(futuresGoogle):
            key = futuresGoogle[future]
            for suggestion in future.result():
                resultList.append([key, suggestion])

    # Convert the results to a dataframe
    suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
    del resultList

    #if we have old results read them
    try:
        suggestion_df=pd.read_csv("keyword_suggestions.csv")
        
    except:
        suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
    
    suggestionCommon_list=[]
    suggestionNew_list=[]
    for keyword in suggestion_new["Keyword"].unique():
        new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
        old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
        newSuggestion=set(new_df["Suggestion"].to_list())
        oldSuggestion=set(old_df["Suggestion"].to_list())
        commonSuggestion=list(newSuggestion & oldSuggestion)
        new_Suggestion=list(newSuggestion - oldSuggestion)
         
        for suggest in commonSuggestion:
            suggestionCommon_list.append([dateTimeObj,keyword,suggest])
        for suggest in new_Suggestion:
            suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
    
    #new keywords
    newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
    #shared keywords with date update
    commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
    merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
    merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
    merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
    del merge["last_seen_x"]
    del merge["Keyword_y"]
    
    #merge old results with new results
    frames = [merge, newSuggestion_df]
    keywords_df =  pd.concat(frames, ignore_index=True, sort=False)
    # Save dataframe as a CSV file
    keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
    keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])   
    keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
    keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
    keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
    keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
    keywords_df.to_csv('keyword_suggestions.csv', index=False)

# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file

Íoslódáil Script Python