Cortex AI 오디오¶
Cortex AI 오디오는 고급 LLM 기반 오디오 처리 기능을 제공하며, 다음을 포함합니다.
전사: 말한 내용을 텍스트로 변환합니다.
화자 식별: 다중 화자 오디오 파일의 각 부분에서 누가 말하고 있는지 판단합니다.
타임스탬프 추출: 발화된 각 단어의 타임스탬프를 식별합니다.
이러한 기능은 AI_TRANSCRIBE 함수를 통해 사용할 수 있습니다. AI_TRANSCRIBE는 Snowflake 내에서 관리 및 호스팅되므로, 번거로운 설정이나 인프라 관리 없이도 오디오 처리를 데이터 워크플로에 손쉽게 통합할 수 있습니다.
참고
The AI_TRANSCRIBE function also processes audio tracks in video files.
AI_TRANSCRIBE¶
AI_TRANSCRIBE is a fully managed SQL function that transcribes audio and video files stored in a stage, extracting text, timestamps, and speaker information. See Create stage for media files for information on creating a stage suitable for storing files for processing by AI_TRANSCRIBE.
Under the hood, AI_TRANSCRIBE orchestrates optimized AI models for transcription and speaker diarization, processing audio files of up to two hours in length. AI_TRANSCRIBE is horizontally scalable, allowing efficient batch processing by processing multiple files at the same time. Audio can be processed directly from object storage to avoid unnecessary data movement.
기본적으로 AI_TRANSCRIBE는 오디오 파일을 깔끔하고 읽기 쉬운 텍스트로 변환합니다. 또한 각 단어 또는 화자 변경에 대한 타임스탬프를 추출하기 위해 타임스탬프 세분성을 지정할 수 있습니다. 단어 단위 타임스탬프는 자막과 같은 애플리케이션이나 사용자가 대본에서 단어를 클릭하여 오디오의 특정 부분으로 이동하도록 하는 데 유용합니다. 화자 수준 타임스탬프는 회의, 인터뷰 또는 전화 통화에서 누가 어떤 말을 했는지 파악하는 데 유용합니다.
타임스탬프 세분성 모드 |
결과 |
|---|---|
기본값 |
한 번에 전체 오디오 파일 전사 |
단어 |
각 단어에 대한 타임스탬프가 포함된 전사 |
화자 |
화자 변경 시마다 화자와 타임스탬프 표시 |
지원되는 언어¶
AI_TRANSCRIBE는 다음 언어를 지원하며, 자동으로 감지합니다. 파일은 여러 지원 언어를 포함할 수 있습니다.
참고
언어 감지를 사용하려면 파일 시작 후 5초 이내에 오디오가 시작되어야 합니다. 최상의 결과를 위해 업로드 전에 불필요한 무음 부분을 잘라내세요.
아랍어
불가리아어
광둥어
카탈루냐어
중국어
체코어
네덜란드어
영어
프랑스어
독일어
그리스어
Hebrew
헝가리어
인도네시아어
이탈리아어
일본어
한국어
라트비아어
Norwegian
폴란드어
포르투갈어
루마니아어
러시아어
세르비아어
슬로베니아어
스페인어
스웨덴어
태국어
터키어
우크라이나어
Supported media formats¶
AI_TRANSCRIBE supports the following audio and video file formats:
Audio |
FLAC, MP3, MP4, OGG, WAV, WEBM |
|---|---|
Video |
MKV, MP4, OGV, WEBM |
Video files must contain at least one audio track in FLAC, MP3, OPUS, VORBIS, or WAV format.
예¶
텍스트 전사¶
다음 예제는 financial_consultation 스테이지에 저장된 :download:`오디오 파일</samples/cortex/audio/consultation.wav>`을 전사하고, 전체 파일의 텍스트 대본을 반환합니다. :doc:`TO_FILE 함수</sql-reference/functions/to_file>`는 스테이징된 파일을 파일 참조로 변환합니다.
SELECT AI_TRANSCRIBE(TO_FILE(
'@financial_consultation', 'consultation.wav'));
응답:
{"audio_duration":321.78,"text":"Good afternoon, Robert. Thanks for calling in
today. I understand you had some concerns about your portfolio you wanted to
discuss. Yes, I'm really worried. I've been watching the news and the market's
been all over the place lately. I'm thinking maybe I should just sell
everything, all my stocks and mutual funds and put it all in bonds or CDs. At
least then I could sleep at night. I can definitely understand that concern,
Robert. Market volatility can be unsettling, especially when you're seeing
those daily swings in the headlines. Before we talk about any major moves, can
you help me understand what specifically is driving this anxiety? Is it the
recent tech sector pullback or something more general? It's everything. I'm 52
years old and I keep thinking about what happened in 2008. I lost so much then
and I'm worried we're heading for another crash with this new administration. I
can't afford to lose my retirement savings. Those are absolutely valid
concerns, and I appreciate you sharing that context. That was a really
challenging time for everyone. Let me ask you this. When we last reviewed your
portfolio in March, we had you allocated at about 70% equities and 30% bonds,
correct? And your target retirement age is still 62%. That's right. But
honestly, 70% in stocks feels way too risky right now. I'm thinking more like
20% stocks, 80% bonds, maybe even less in stocks. I understand that instinct,
Robert. Let's walk through this together. First, I want to remind you of
something important. Your current portfolio is already designed with volatility
in mind. You're not in individual stocks. You're in diversified index funds and
some actively managed funds across different sectors and even international
markets. but they're still going down. My quarterly statement showed I was down
8% this quarter alone. You're absolutely right, and that's painful to see, but
let's put this in perspective. Over the past 12 months, even with this recent
volatility, your portfolio is still up about 3%. The market has given back some
gains, but we're not in crisis territory. Remember, we built your allocation
specifically because you have 10 years until retirement. That time horizon is
actually your biggest asset here. So you're saying I should just do nothing?
Not exactly nothing, but I am suggesting we don't make dramatic changes based
on short-term market movements. However, I do hear your concern about risk
tolerance. What if we made a smaller adjustment? Instead of going to 20%
stocks, what if we moved to 60% stocks and 40% bonds? That would reduce your
equity exposure by 10%, which might help you sleep better, but wouldn't take
you completely out of the growth potential you need for retirement. That
actually sounds more reasonable, but I'm still worried about losing more
money. I understand completely. Let me ask you this. What's your bigger worry,
the volatility of the next year, two or two, or having enough money to retire
comfortably at 62? Because if we get too conservative now, inflation alone
could erode your purchasing power over the next decade. I didn't really thought
about inflation that way. I guess I've been so focused on not losing money that
I forgot about the money I might not make. Exactly. And remember, Robert,
you're not alone in this. I've had this conversation with many clients over the
past few weeks. The ones who stayed disciplined during previous market
downturns are generally glad they did. What if we also set up a plan where we
review your portfolio monthly for the next few months? That way you'll have
regular check-ins and won't feel like you're just riding this out blindly.
Monthly reviews would definitely help. And maybe the 60-40 split is a good
compromise. I just, I don't want to be stupid about this. Overt, wanting to
protect your retirement isn't stupid. It's exactly what you should be thinking
about. The key is making sure we're protecting it in the right way. Staying
invested in a diversified portfolio, even with some volatility, has
historically been the best way to preserve and grow wealth over time. okay, I
think I can live with moving to 60% stocks, but if things get really bad... If
things get really bad, we'll talk again. That's what I'm here for. And
remember, we'll be reviewing this monthly anyway. You're not locked into
anything forever. But I do want to emphasize that market timing is incredibly
difficult, even for professionals. The goal isn't to avoid all volatility.
It's to stay invested long enough to benefit from the market's long-term
upward trend. All right, Sarah, let's do the rebalancing to 60-40 and I'll try
to stop checking my account balance every day. It sounds like a solid plan,
Robert. And yes, definitely limit the daily balance checking. That's a recipe
for anxiety. I'll send you some research on historical market recoveries after
our call and we'll schedule our first monthly review for next month. How does
that sound? That sounds good. Thanks for talking me through this, Sarah. I feel
a lot better than when I call. I'm so glad to hear that, Robert. Remember,
staying invested requires patience, but your future self will thank you for it.
I'll have the rebalancing done by tomorrow morning, and you should see the
changes reflected in your account by Thursday. Perfect. Thanks again, Sarah. I
thank you deeply for your patience and understanding. I'll talk to you next
month."}
타임스탬프를 포함한 단어 수준 분할¶
타임스탬프 세분성을 “단어”로 설정하여 발화된 모든 단어에 대한 정확한 타임스탬프를 추출함으로써 검색 및 탐색이 가능한 대본을 생성합니다. :download:`이 오디오 파일</samples/cortex/audio/consultation_3_sp.wav>`은 스페인어로 되어 있습니다.
SELECT AI_TRANSCRIBE(TO_FILE('@financial_consultation', 'consultation_3_sp.wav'),
{'timestamp_granularity': 'word'});
응답:
참고
간결함을 위해 출력이 잘렸습니다. 전체 출력에는 오디오 파일에서 발화된 각 단어에 대한 세그먼트가 포함됩니다.
{
"audio_duration": 150.66,
"segments": [
{
"end": 1.513,
"start": 0.031,
"text": "«Buenos"
},
{
"end": 2.034,
"start": 1.553,
"text": "días,"
},
{
"end": 2.334,
"start": 2.054,
"text": "doña"
},
{
"end": 4.457,
"start": 2.374,
"text": "Esperanza."
},
{
"end": 4.597,
"start": 4.477,
"text": "¿En"
},
{
"end": 4.857,
"start": 4.697,
"text": "qué"
},
{
"end": 5.118,
"start": 4.917,
"text": "puedo"
},
{
"end": 5.518,
"start": 5.178,
"text": "ayudarla"
},
{
"end": 6.5,
"start": 5.578,
"text": "hoy?»"
},
...
{
"end": 146.671,
"start": 146.551,
"text": "Ya"
},
{
"end": 147.234,
"start": 146.732,
"text": "veremos,"
},
{
"end": 147.837,
"start": 147.355,
"text": "Roberto."
},
{
"end": 148.581,
"start": 148.078,
"text": "Gracias"
},
{
"end": 148.822,
"start": 148.661,
"text": "por"
},
{
"end": 149.646,
"start": 148.902,
"text": "tu"
},
{
"end": 150.711,
"start": 150.249,
"text": "ayuda."
}
],
"text": "«Buenos días, doña Esperanza. ¿En qué puedo ayudarla hoy?» «Roberto, quiero
hacer un cambio grande en mi portafolio. Quiero vender todo y compra solo acciones
de Tesla». «¿Tesla? Doña Esperanza, usted tiene 72 años. ¿Por qué quiere poner todo
su dinero en una sola compañía?» «¿Por qué Tesla va a ser el futuro?» Un minuto me
explico que van a dominar los carros eléctricos. Dice que puedo triplicar mi dinero
en dos años. Entiendo que Tesla es una impresión innovador, pero poner todos sus
ajuros en una sola acción es muy arriesgado. ¿Qué pasa si Tesla baja? No va a bajar.
Elon Musk es un genio. Además, mi vecina compró Teslas. Teslas es tres años. Y Aorus
tiene el doble de dinero. Doña Esperanza, su vecina tuvo suerte, pero las yantes
individuales pueden ser muy volátiles. Usted necesita dinero estable para sus gastos
de retiro. Roberto, tengo $400,000 en mi cuenta. Si te la sube como dismi, voy a
tener más de un año. Podré dejarle más dinero a mi familia. Pero también podría
perder la mitad de su dinero o más. Te sabía Jairo 60% antes. No puedo recomendarle
que haga esto. Entonces no me dejas escuchando. Yo sé lo que quiero hacer con mi
dinero. Es mi decisión. Tienes razón, es su dinero. Pero como su asesor tengo que
decir que esto es extremamanda peligroso para alguien de su edad. Eva, no importa.
Quiero tomar este riesgo. Vas a Edom o no. Doña Esperanza, ¿qué tal si compramos
algo de Tesla perronoto? ¿Podríamos poner 10% en Tesla y el resto en versiones más
seguras? No, Roberto, quiero el 100% en Tesla. Si no me ayudas, voy a alcanzar otro
asesor. Que sí lo haga. Está bien, Doña Presanza. Voy a procesar la orden, pero voy
a documentar que fue contra mi recomendación profesional. Perfecto. Hazlo hoy mismo.
Quiero compra antes que suba más. Será ahora. Él considera lo que le estoy diciendo.
Esto puede ser ver muy mal a la vida. Ya veremos, Roberto. Gracias por tu ayuda."
}
화자 인식¶
대화나 회의에서 고유한 화자를 감지하고, 분리하고, 식별하기 위해 타임스탬프 세분성을 “화자”로 설정하세요. 이 예제에서는 두 명의 화자가 등장하여 한 명은 영어로, 다른 한 명은 스페인어로 말하는 :download:`오디오 파일</samples/cortex/audio/consultation_5_mix_es_en.wav>`을 사용합니다.
SELECT AI_TRANSCRIBE(TO_FILE('@financial_consultation', 'consultation_5_mix_es_en.wav'),
{'timestamp_granularity': 'speaker'});
응답:
참고
간결함을 위해 출력이 잘렸습니다. 전체 출력에는 오디오 파일의 각 대화 “턴”에 대한 세그먼트가 포함됩니다.
{
"audio_duration": 208.66,
"segments": [
{
"end": 3.076,
"speaker_label": "SPEAKER_00",
"start": 0.031,
"text": "Good afternoon, this is Aaliyah Johnson from Secure Financial Services."
},
{
"end": 4.297,
"speaker_label": "SPEAKER_02",
"start": 3.196,
"text": "How can I help you today?"
},
{
"end": 7.182,
"speaker_label": "SPEAKER_02",
"start": 5.139,
"text": "Hola, necesito ayuda con mis inversiones."
},
{
"end": 11.528,
"speaker_label": "SPEAKER_02",
"start": 7.482,
"text": "Estoy muy preocupada porque he perdido mucho dinero y no sé qué hacer."
},
{
"end": 14.132,
"speaker_label": "SPEAKER_02",
"start": 12.289,
"text": "I'm sorry, I'm not understanding."
},
{
"end": 15.795,
"speaker_label": "SPEAKER_02",
"start": 14.553,
"text": "Do you speak English?"
},
...
{
"end": 189.169,
"speaker_label": "SPEAKER_02",
"start": 185.841,
"text": "Es muy difícil entender estas cosas en inglés."
},
{
"end": 192.326,
"speaker_label": "SPEAKER_01",
"start": 190.178,
"text": "Por supuesto, señora Ramírez."
},
{
"end": 197.145,
"speaker_label": "SPEAKER_01",
"start": 192.788,
"text": "Es muy importante que entienda completamente sus opciones."
},
{
"end": 203.229,
"speaker_label": "SPEAKER_01",
"start": 197.165,
"text": "Voy a hacer los cambios hoy mismo y la llamaré la próxima semana para ver cómo se siente."
},
{
"end": 205.759,
"speaker_label": "SPEAKER_02",
"start": 203.891,
"text": "Muchísimas gracias, María."
},
{
"end": 208.71,
"speaker_label": "SPEAKER_02",
"start": 206.18,
"text": "Me siento mucho más tranquila ahora."
}
],
"text": "Good afternoon, this is Aaliyah Johnson from Secure Financial Services.
How can I help you today? Hola, necesito ayuda con mis inversiones. Estoy muy
preocupada porque he perdido mucho dinero y no sé qué hacer. I'm sorry, I'm not
understanding. Do you speak English? Un poquito, pero es muy difícil para mí. Aquí
hay alguien que habla español, ¿ok? Es muy importante. He perdido miles de dólares.
I'm really sorry, but I don't speak Spanish. Let me see. I think we might have
someone who speaks Spanish, but they're not available right now. ¿Cuándo pueden
ayudarme? Necesito hablar con a lguien hoy. Mi esposo está muy enojado y quiere que
vendamos todo. I understand you need someone who speaks Spanish. Let me check if
Maria is available. She's our Spanish-speaking advisor. Can you hold for just a
moment? No entiendo. Mañana. Pero necesito ayuda ahora. ¿No hay nadie más? I am
going to transfer you to Maria right now. She'll be able to help you with your
investment concerns. Hola, soy María González. Entiendo que necesita ayuda con sus
inversiones. ¿Cómo está usted? ¡Ay, qué alivio! Sí, estoy muy preocupada. He
perdido casi 20.000 dólares en las últimas semanas y mi esposo quiere que vendamos
todo. Comprendo perfectamente su preocupación, señora Ramírez. Perder dinero es muy
estresante. Cuénteme un poco más sobre su situación. ¿Qué tipo de inversiones
tiene? Tengo fondos mutuos y algunas acciones. Todo está bajando mucho. Mi esposo
dice que es mejor tener el dinero en el banco, pero yo no estoy segura. Es natural
sentirse nerviosa cuando el mercado está volátil. Pero antes de tomar decisiones
importantes, vamos a revisar su situación completa. ¿Cuántos años tiene usted y
cuándo planea retirarse? Tengo 55 años y quiero retirarme a los 65, pero con estas
pérdidas no sé si voy a poder. Señora Ramírez, usted todavía tiene 10 años hasta el
retiro. Eso es tiempo suficiente para que sus inversiones se recuperen. El mercado
siempre tiene altibajos, pero históricamente se ha recuperado. ¿Pero qué pasa si no
se recupera esta vez? No puedo perder más dinero. Entiendo su miedo. ¿Qué le parece
si hacemos algunos ajustes para que se sienta más cómoda? Podemos mover parte de su
dinero a inversiones más conservadoras, como bonos. Eso suena mejor. No quiero
arriesgar todo, pero tampoco quiero perder la oportunidad de crecer mi dinero.
Perfecto. Vamos a encontrar un equilibrio. ¿Qué tal si movemos el 40% de sus
acciones a bonos? Así tendrá menos riesgo, pero todavía podrá crecer su dinero para
el retiro. Sí, eso me hace sentir mucho mejor. Gracias por explicarme todo en
español. Es muy difícil entender estas cosas en inglés. Por supuesto, señora
Ramírez. Es muy importante que entienda completamente sus opciones. Voy a hacer los
cambios hoy mismo y la llamaré la próxima semana para ver cómo se siente. Muchísimas
gracias, María. Me siento mucho más tranquila ahora."
}
Use with other AI Functions¶
Call transcript analysis¶
You can pass the output of AI_TRANSCRIBE to other AI Functions for further processing. For example, you can use
AI_SUMMARIZE to summarize the transcription, or AI_CLASSIFY to classify the content of the transcription. This example
uses AI_SENTIMENT and AI_COMPLETE to analyze the text transcribed from
customer call audio and provide sentiment on four dimensions
and an assessment of the agent.
참고
AI_SENTIMENT는 텍스트만을 분석하며, 음성 톤과 같은 음성 특성은 고려하지 않습니다.
WITH transcriptions AS
( SELECT TO_VARCHAR (AI_TRANSCRIBE(TO_FILE('@financial_consultation',
'consultation_1.wav'))) AS transcribed_call )
SELECT
AI_SENTIMENT(transcribed_call, ['Professionalism', 'Resolution',
'Wait Time', 'Market Conditions']) AS call_sentiment,
AI_COMPLETE ('claude-4-opus', CONCAT ('Summarize how the agent can improve in 50 words',
transcribed_call)) AS agent_assessment
FROM transcriptions
AI_SENTIMENT 응답:
{
"categories": [
{
"name": "overall",
"sentiment": "negative"
},
{
"name": "Market Conditions",
"sentiment": "negative"
},
{
"name": "Professionalism",
"sentiment": "negative"
},
{
"name": "Resolution",
"sentiment": "negative"
},
{
"name": "Wait Time",
"sentiment": "unknown"
}
]
}
AI_COMPLETE 응답:
"The agent needs significant improvement in empathy, active listening, and client-centered communication. Instead of
dismissing concerns and using condescending language, they should validate emotions, explain market conditions
professionally, present multiple options, and guide clients through informed decision-making while respecting their
risk tolerance and personal circumstances."
Video transcript analysis¶
The following example transcribes a video file stored in the podcast_videos_S3 stage,
SELECT AI_TRANSCRIBE(TO_FILE( '@podcast_videos_S3', 'podcast-interview.mp4'));
응답:
{
"audio_duration": 5423.744,
"text": "Welcome to the New York Times Popcast, your deepest duende of music news and criticism. I'm John Caramonica, and I'm the critic. I'm Joe Cascarelli, and I'm the reporter. I'm Rosalía and I'm here today with you guys. Yes. Thank you so much for being here. Like literally on some days, Jo. Some days. On some days, I think, is this person the only good pop star?
...
Thank you for being here. Loved. Every episode of Popcast is at nytimes.com slash popcast. We're on YouTube at Popcast. Subscribe. We're on Instagram and TikTok at Popcast. Tap that like. Tap that follow. Tap in. Don't tap out. Credits and links and bio. We'll be back next week. Yes. Invite me anytime to eat more snacks, please. I lost my hands in Jerez"
}
Once you have the transcript, you can use AI_COMPLETE to perform additional analysis. This example identifies retail brands mentioned in the conversation for use in advertising or sponsorship analytics.
SELECT
AI_COMPLETE('claude-sonnet-4-5',
PROMPT('Return a list of any Retail Brands mentioned in this podcast {0}',
TO_VARCHAR(transcription_results))) as brands_identified
FROM podcast_video_transcription;
Response
Retail Brands Mentioned in Podcast
Based on the transcript analysis, the following brands were identified:
Calvin Klein — Mentioned in relation to Rosalía’s commercial appearance
Kinder Bueno — Cited as one of Rosalía’s favorite snacks.
Nutella — Referenced as a preferred treat.
Nestlé — Mentioned as the manufacturer of Milky Bar ice cream bites.
Nongshim — Korean snack brand discussed during the tasting segment.
Cap'n Crunch — Referenced for its scent similarity to Korean snacks.
Doritos — Mentioned by one of the hosts while discussing snack collections.
비용 고려 사항¶
Billing for all AI Functions is based on token consumption. For transcription, each second of audio processed is 50 tokens, regardless of language or segmentation method. A full hour of audio is therefore 180,000 tokens. Assuming that processing a million tokens costs 1.3 credits, and that Snowflake credits cost US $3 each, each hour of audio processed costs about US $0.702. This estimate is subject to change. For current pricing information, see the Snowflake Service Consumption Table.
참고
AI_TRANSCRIBE의 최소 청구 기간은 1분입니다. 1분 미만의 파일도 처리되지만, 1분 단위로 청구됩니다. 대량의 짧은 오디오 파일을 효율적으로 처리하려면, 이를 단일 파일로 일괄 처리하고 타임스탬프를 사용하여 결과 전사에서 각 원본 파일의 시작과 끝을 식별할 수도 있습니다.