[Python] How to turn tensor type to original text (before tokenized) in Pytorch

Discussão em 'Python' iniciado por Stack, Abril 28, 2025.

  1. Stack

    Stack Membro Participativo

    for example, a tensor type data below is tokenized by a kind of English tokenizer.

    tensor([[ 2992, 1852, 9439, ..., 2610, 1704, 29189],
    [ 1852, 9439, 7, ..., 1704, 29189, 23223],
    [ 9439, 7, 2367, ..., 29189, 23223, 838],
    ...,
    [ 12, 7469, 28844, ..., 2973, 16, 73],
    [ 7469, 28844, 28469, ..., 16, 73, 735],
    [28844, 28469, 191, ..., 73, 735, 4482]])


    how to transform it to original English text? (using Pytorch)

    Continue reading...

Compartilhe esta Página