Skip to content

Transform audio recordings to embeddings

Source: R/2_1_talkEmbed.R
talkEmbed.Rd

Transform audio recordings to embeddings

Usage

talkEmbed(
  talk_filepaths,
  model = "openai/whisper-small",
  audio_transcriptions = "None",
  use_decoder = FALSE,
  tokenizer_parallelism = FALSE,
  model_max_length = "None",
  device = "cpu",
  hg_gated = FALSE,
  hg_token = "",
  trust_remote_code = FALSE,
  logging_level = "warning"
)

Arguments

talk_filepaths

(string) path to a video file (.wav/) list of audio filepaths, each is embedded separately

model

shortcut name for Hugging Face pretained model. Full list https://huggingface.co/transformers/pretrained_models.html

audio_transcriptions

(strings) audio_transcriptions : list (optional) list of audio transcriptions, to be used for Whisper's decoder-based embeddings

use_decoder

(boolean) whether to use Whisper's decoder last hidden state representation (Note: audio_transcriptions must be provided if this option is set to true)

tokenizer_parallelism

(boolean) whether to use device parallelization during tokenization.

model_max_length

(integer) maximum length of the tokenized text

device

(string) name of device: 'cpu', 'gpu', or 'gpu:k' where k is a specific device number

hg_gated

(boolean) set to True if the model is gated

hg_token

(string) the token to access the gated model got in huggingface website

trust_remote_code

(boolean) use a model with custom code on the Huggingface Hub.

logging_level

(string) Set logging level, options: "critical", "error", "warning", "info", "debug".

Value

A tibble with embeddings.

See also

talkText.

Examples

# Transform audio recordings in the example dataset:
# voice_data (included in talk-package), to embeddings.
if (FALSE) { # \dontrun{
wav_path <- system.file("extdata/",
"test_short.wav",
package = "talk")

talk_embeddings <- talkEmbed(
wav_path
)
talk_embeddings
} # }

GitHub