Recreate singing voice using AI

sixwest

Hello:

I was interested in knowing if it was possible using AI software to recreate a specific singing voice. I have several samples available and would like the voice recreated to basically "sing another song". Is that possible using AI? Thanks for all replies...

qryss

Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Cambler

@qryss said:
Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Wow I didn't know of this. Impressive but limited to a specific type of voice. Terms and conditions are untenable though, in my opinion.

qryss

@Cambler said:

@qryss said:
Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Wow I didn't know of this. Impressive but limited to a specific type of voice. Terms and conditions are untenable though, in my opinion.

There are quite a few voices available - although Solaria is the most recent and the most realistic. Kevin is not far behind.

In fact, there is a lot of control of the voice available, both in terms of expression and articulation but also in terms of basic voice character - there is a thread on vi-control.net with some amazing examples - but, yes: Solaria is just one voice and that will limit its application for sure.

Not sure what you mean about T&Cs being untenable. I may be wrong but I’m pretty sure the paid version has no restrictions on what you do with its output. Happy to be enlightened!

Cambler

@qryss said:

@Cambler said:

@qryss said:
Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Wow I didn't know of this. Impressive but limited to a specific type of voice. Terms and conditions are untenable though, in my opinion.

There are quite a few voices available - although Solaria is the most recent and the most realistic. Kevin is not far behind.

In fact, there is a lot of control of the voice available, both in terms of expression and articulation but also in terms of basic voice character - there is a thread on vi-control.net with some amazing examples - but, yes: Solaria is just one voice and that will limit its application for sure.

Not sure what you mean about T&Cs being untenable. I may be wrong but I’m pretty sure the paid version has no restrictions on what you do with its output. Happy to be enlightened!

Sorry, my comment on the limitations of the voice are probably a little unfair seeing as I haven't used it, just going off the demos. In regards to the T&Cs, it's this that stuck out to me,

"If the user of Synthesizer V SOLARIA is a legal entity like a corporation or business who wishes to use Synthesizer V SOLARIA in public works of any kind, an additional license must be secured from Eclipsed Sounds, with restrictions based on the type of usage. This is particularly for any usage in corporate advertisement or advertising materials."

So as I understand this, if you create a cracking pop song, using vocals from Solaria, you do not have full control over your composition and if a licensing opportunity came up for film, tv, advertising you would have to seek permission and an additional license from Eclipsed Sounds. This is basically a synthesiser so that is a far reach in my book. Imagine if Moog had sought the same?

Regardless, the tech is very impressive and an indication that this area of synthesis will become more prominent. I imagine we'll start to see it from bigger companies soon, like NI et al.

farfromsubtle

Synthesizer V voices are certainly the best sounds and most controllable right now. Glottal effects are needed to make them usable for more genres but those are coming.

Custom singing models that can be trained on “few-shot” learning are an inevitability as it is already happening with incredible results from 15.ai which is especially exciting as that those models in particular give you different “takes” on every generation, which feels like working with an actual performer.

qryss

I missed that about the licensing, @Cambler. I’ll see if I can find anyone who has sought such a licence and what it cost them.

By the way, I have this software but I use it to make guide vocals for a real singer so no licensing issues should arise.

Here’s another example.

qryss

@Cambler said:

@qryss said:

@Cambler said:

@qryss said:
Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Wow I didn't know of this. Impressive but limited to a specific type of voice. Terms and conditions are untenable though, in my opinion.

There are quite a few voices available - although Solaria is the most recent and the most realistic. Kevin is not far behind.

In fact, there is a lot of control of the voice available, both in terms of expression and articulation but also in terms of basic voice character - there is a thread on vi-control.net with some amazing examples - but, yes: Solaria is just one voice and that will limit its application for sure.

Not sure what you mean about T&Cs being untenable. I may be wrong but I’m pretty sure the paid version has no restrictions on what you do with its output. Happy to be enlightened!

Sorry, my comment on the limitations of the voice are probably a little unfair seeing as I haven't used it, just going off the demos. In regards to the T&Cs, it's this that stuck out to me,

"If the user of Synthesizer V SOLARIA is a legal entity like a corporation or business who wishes to use Synthesizer V SOLARIA in public works of any kind, an additional license must be secured from Eclipsed Sounds, with restrictions based on the type of usage. This is particularly for any usage in corporate advertisement or advertising materials."

So as I understand this, if you create a cracking pop song, using vocals from Solaria, you do not have full control over your composition and if a licensing opportunity came up for film, tv, advertising you would have to seek permission and an additional license from Eclipsed Sounds. This is basically a synthesiser so that is a far reach in my book. Imagine if Moog had sought the same?

Regardless, the tech is very impressive and an indication that this area of synthesis will become more prominent. I imagine we'll start to see it from bigger companies soon, like NI et al.

I’m not a lawyer but I believe the clause preceding the one you quoted is there to distinguish between a human entity and a corporate entity. That would mean that a ‘person’ - even if they operate as a company - is exempt from needing that licence. It looks to me that they are trying to make money from corporates (who presumably are less likely to have an issue with this) but not from people who are, well, people.

How that stands up in court I have no idea and would depend on the jurisdiction but I suggest the likes of us here are mostly pretty safe.

2. Natural person users can use the synthesized voice generated by or role copyright image included in this product under non-commercial or commercial situations for activities, creation, work spreading, publication, etc.

I get the synthesizer plugin analogy but right now there are very few companies who can do AI voices this well so I guess they are protecting themselves - it’s cheap software but they want their slice if they can get it.

qryss

Someone on VI-Control asked Dreamtronics for clarification and got this as a response - if it helps.

Our license states that any individual person who wishes to use SOLARIA commercially may do so without limit, and that there is only the need to contact us for an additional license if a corporation, business, or other legal entity (e.g. Google, Crypton Future Media, Dreamtonics) wants to utilize the voice in a public work or advertisement. If you are an independent musician who wishes to utilize the SOLARIA voice library, you are free to release any of your works commercially with her. I hope that this clarified your question.

NeuM

@Cambler said:

@qryss said:
Solaria - a voice for Synthesizer V - is probably the best AI voice at present but it doesn’t repurpose samples, it uses phonemes to sing words you type in.

Here’s an example but it’s not available for iOS.

Wow I didn't know of this. Impressive but limited to a specific type of voice. Terms and conditions are untenable though, in my opinion.

Here's a presentation from the company that makes the software:

GovernorSilver

Oh wow, Solaria sounds better than any of the Vocaloid plugins I listened to - in demos of course.

Cambler

@qryss said:
I missed that about the licensing, @Cambler. I’ll see if I can find anyone who has sought such a licence and what it cost them.

By the way, I have this software but I use it to make guide vocals for a real singer so no licensing issues should arise.

Here’s another example.

Very cool! I hadn't thought of this for use with cover songs but it's ideal for the right kind of song. Now I feel myself reaching for my wallet. Stop me!

Cambler

delete

Cambler

@Cambler said:

@qryss said:
Someone on VI-Control asked Dreamtronics for clarification and got this as a response - if it helps.

Our license states that any individual person who wishes to use SOLARIA commercially may do so without limit, and that there is only the need to contact us for an additional license if a corporation, business, or other legal entity (e.g. Google, Crypton Future Media, Dreamtonics) wants to utilize the voice in a public work or advertisement. If you are an independent musician who wishes to utilize the SOLARIA voice library, you are free to release any of your works commercially with her. I hope that this clarified your question.

Thanks @qryss. The reason I'm wary is that I see a great use case for this being around the production of music for licensing. One has to be nimble in these circumstances and if an entity wants to license a work of yours, you ideally don't want to have to go to a third party to ask permission. I'm not really sure why they have this clause? I wonder when they would say no? Must follow up on the VI forum. Do you have the link to that specific thread? Thanks for all the tips!

qryss

@Cambler said:

@Cambler said:

@qryss said:
Someone on VI-Control asked Dreamtronics for clarification and got this as a response - if it helps.

Our license states that any individual person who wishes to use SOLARIA commercially may do so without limit, and that there is only the need to contact us for an additional license if a corporation, business, or other legal entity (e.g. Google, Crypton Future Media, Dreamtonics) wants to utilize the voice in a public work or advertisement. If you are an independent musician who wishes to utilize the SOLARIA voice library, you are free to release any of your works commercially with her. I hope that this clarified your question.

Thanks @qryss. The reason I'm wary is that I see a great use case for this being around the production of music for licensing. One has to be nimble in these circumstances and if an entity wants to license a work of yours, you ideally don't want to have to go to a third party to ask permission. I'm not really sure why they have this clause? I wonder when they would say no? Must follow up on the VI forum. Do you have the link to that specific thread? Thanks for all the tips!

There’s a mega thread full of practical tips here: https://vi-control.net/community/threads/synthesizer-v-vocaloid-haters-might-want-to-check-this.115973/

Cambler

@qryss said:

@Cambler said:

@Cambler said:

@qryss said:
Someone on VI-Control asked Dreamtronics for clarification and got this as a response - if it helps.

Our license states that any individual person who wishes to use SOLARIA commercially may do so without limit, and that there is only the need to contact us for an additional license if a corporation, business, or other legal entity (e.g. Google, Crypton Future Media, Dreamtonics) wants to utilize the voice in a public work or advertisement. If you are an independent musician who wishes to utilize the SOLARIA voice library, you are free to release any of your works commercially with her. I hope that this clarified your question.

Thanks @qryss. The reason I'm wary is that I see a great use case for this being around the production of music for licensing. One has to be nimble in these circumstances and if an entity wants to license a work of yours, you ideally don't want to have to go to a third party to ask permission. I'm not really sure why they have this clause? I wonder when they would say no? Must follow up on the VI forum. Do you have the link to that specific thread? Thanks for all the tips!

There’s a mega thread full of practical tips here: https://vi-control.net/community/threads/synthesizer-v-vocaloid-haters-might-want-to-check-this.115973/

Magic, thanks so much. I've read through the fine details of the agreement and I think it's probably ok. I'm just not sure what the definition of a "legal entity" is. I'm sure all working musicians are set up as some form of legal business, wether a sole trader or partnership or possibly a limited company or the like. Anyway they do say that a "legal person" can commercially exploit work made with the software for advertising, film, tv etc. Noting that I'm not a lawyer and that is not advice

Now I want to buy it. DAMN THIS FORUM (with love)!