[Python] How to skip data already imported using Python

Discussão em 'Python' iniciado por Stack, Outubro 24, 2024.

  1. Stack

    Stack Membro Participativo

    I am importing data from a csv file into a database using an API. I've already run the script below and a little less than half of the data was imported. Something caused the API to stop and now I am looking to run the script again. I've tested a few lines from the csv file and data that has already been imported is being duplicated. Is there a way to modify the script so that imported data is skipped? I was thinking some type of if statement, but I am not quite sure. Any suggestions?

    def opencsv():
    '''This function opens a csv file'''
    input_csv = 'agentlinks.csv'
    file = open(input_csv, 'r')
    csvin = csv.reader(file)
    #Skip header row
    next(csvin, None)
    return csvin

    def addUriLink():
    """Add URI link to record."""
    api_url, headers = as_login.login()
    csvfile = opencsv()
    for row in csvfile:
    repo_id = row[1]
    resource_id = row[2]
    value = row[3]
    resource_uri='repositories/' + repo_id + '/resources/' + str(resource_id)
    resource_json = requests.get(api_url + '/repositories/' + repo_id + '/resources/' + str(resource_id), headers=headers).json()
    updatedRecord = resource_json
    agents = updatedRecord['linked_agents']
    agent = {}
    agent['ref'] = value
    agent['role'] = 'subject'
    if agent not in agents:
    agents.append(agent)
    updatedRecord['linked_agents'] = agents
    print(updatedRecord['linked_agents'])
    updatedRecord = json.dumps(updatedRecord)
    print(api_url + resource_uri)
    post = requests.post(api_url + '/repositories/' + repo_id + '/resources/' + str(resource_id), headers=headers, data=updatedRecord).json()
    print(post)
    else:
    print('no update')
    log = open("addAgentsLinks.txt", "a")
    print(updatepost, file=log)
    addUriLink()

    Continue reading...

Compartilhe esta Página