swift - Converting AudioBuffer to CMSampleBuffer with accurate CMTime -
the goal here create mp4 file via video through avcapturedataoutput , audio recorded coreaudio. send cmsamplebuffers of both avassetwriter has accompanying avassetwriterinput(avmediatypevideo) , avassetwriterinput(avmediatypeaudio)
my audio encoder copies audiobuffer new cmsamplebuffer passes avassetwriterinput(avmediatypeaudio). example how conversion of audiobuffer cmsamplebuffer done. converstion cmsamplebuffer
long story short, not work. video shows no audio.
but, if comment out video encoding, audio written file , audible.
that tells me experience timing issue. converstion cmsamplebuffer show
cmsampletiminginfo timing = { cmtimemake(1, 44100.0), kcmtimezero, kcmtimeinvalid }; it produces time cmtimecopydescription of {0/1 = 0.000} seems wrong me. tried keeping track of frames rendered , passing framecount time value , samplerate time scale this
cmsampletiminginfo timing = { cmtimemake(1, 44100.0), cmtimemake(self.framecount, 44100.0), kcmtimeinvalid }; but no dice. nicer looking cmsampletiminginfo {107520/44100 = 2.438}, still no audio in file.
the video cmsamplebuffer produces {65792640630624/1000000000 = 65792.641, rounded}. tells me avcapturevideooutput has time scale of 1 billion, nanoseconds. , guest time value device time. cant find info avcapturevideooutput uses.
anyone have helpful guidance? on right track?
heres conversion
cmsamplebufferref buff = malloc(sizeof(cmsamplebufferref)); cmformatdescriptionref format = null; self.framecount += innumberframes; cmtime presentationtime = cmtimemake(self.framecount, self.pcmasbd.msamplerate); audiostreambasicdescription audioformat = self.pcmasbd; checkerror(cmaudioformatdescriptioncreate(kcfallocatordefault, &audioformat, 0, null, 0, null, null, &format), "could not create format audiostreambasicdescription"); cmsampletiminginfo timing = { cmtimemake(1, self.pcmasbd.msamplerate), presentationtime, kcmtimeinvalid }; checkerror(cmsamplebuffercreate(kcfallocatordefault, null, false, null, null, format, (cmitemcount)innumberframes, 1, &timing, 0, null, &buff), "could not create cmsamplebufferref"); checkerror(cmsamplebuffersetdatabufferfromaudiobufferlist(buff, kcfallocatordefault, kcfallocatordefault, 0, audiobufferlist), "could not set data in cmsamplebufferref"); [self.delegate didrenderaudiosamplebuffer:buff]; cfrelease(buff); and assetwriters create
func createvideoinputwriter()->avassetwriterinput? { let numpixels = int(self.size.width * self.size.height) let bitsperpixel:int = 11 let bitrate = int64(numpixels * bitsperpixel) let fps:int = 30 let settings:[nsobject : anyobject] = [ avvideocodeckey : avvideocodech264, avvideowidthkey : self.size.width, avvideoheightkey : self.size.height, avvideocompressionpropertieskey : [ avvideoaveragebitratekey : nsnumber(longlong: bitrate), avvideomaxkeyframeintervalkey : nsnumber(integer: fps) ] ] var assetwriter:avassetwriterinput! if self.mainassetwriter.canapplyoutputsettings(settings, formediatype:avmediatypevideo) { assetwriter = avassetwriterinput(mediatype:avmediatypevideo, outputsettings:settings) assetwriter.expectsmediadatainrealtime = true if self.mainassetwriter.canaddinput(assetwriter) { self.mainassetwriter.addinput(assetwriter) } } return assetwriter; } func createaudioinputwriter()->avassetwriterinput? { let settings:[nsobject : anyobject] = [ avformatidkey : kaudioformatmpeg4aac, avnumberofchannelskey : 2, avsampleratekey : 44100, avencoderbitratekey : 64000 ] var assetwriter:avassetwriterinput! if self.mainassetwriter.canapplyoutputsettings(settings, formediatype:avmediatypeaudio) { assetwriter = avassetwriterinput(mediatype:avmediatypeaudio, outputsettings:settings) assetwriter.expectsmediadatainrealtime = true if self.mainassetwriter.canaddinput(assetwriter) { self.mainassetwriter.addinput(assetwriter) } else { let error = nserror(domain:cmhdfileencoder.domain, code:cmhdfileencodererrorcode.cantaddinput.rawvalue, userinfo:nil) self.errordelegate.hdfileencodererror(error) } } else { let error = nserror(domain:cmhdfileencoder.domain, code:cmhdfileencodererrorcode.cantapplyoutputsettings.rawvalue, userinfo:nil) self.errordelegate.hdfileencodererror(error) } return assetwriter }
of course, had problem 2 weeks, posted question on friday night, , found solution monday morning.
the research came across put me on right track...
1000000000 timescale nano seconds. timevalue has nanoseconds of devices absolute time.
this post explains better can - mach time
i ended using code fix it
cmsamplebufferref buff = malloc(sizeof(cmsamplebufferref)); cmformatdescriptionref format = null; audiostreambasicdescription audioformat = self.pcmasbd; checkerror(cmaudioformatdescriptioncreate(kcfallocatordefault, &audioformat, 0, null, 0, null, null, &format), "could not create format audiostreambasicdescription"); uint64_t time = intimestamp->mhosttime; /* convert nanoseconds */ time *= info.numer; time /= info.denom; cmtime presentationtime = cmtimemake(time, kdevicetimescale); cmsampletiminginfo timing = { cmtimemake(1, self.pcmasbd.msamplerate), presentationtime, kcmtimeinvalid }; checkerror(cmsamplebuffercreate(kcfallocatordefault, null, false, null, null, format, (cmitemcount)innumberframes, 1, &timing, 0, null, &buff), "could not create cmsamplebufferref"); checkerror(cmsamplebuffersetdatabufferfromaudiobufferlist(buff, kcfallocatordefault, kcfallocatordefault, 0, audiobufferlist), "could not set data in cmsamplebufferref");
Comments
Post a Comment