Comments (6)
Hey, this isn't being ignored - I've been talking to eng to figure out what's going on to normalize the probability (getting [0,1] instead of the positive/negative large numbers). Waiting for someone to get back next week to dig deeper into the issue.
Thanks!
from mediapipe-samples.
Splendid!... waiting for the answer soon :)
from mediapipe-samples.
So this API is going to change a little bit for the next preview release (nothing significant, but we'll have an updated sample when it's ready), but if you're just trying to make confidence masks work for 0.1.0-alpha-6, this is what I did to make it work:
Helper:
in setupImageSegmenter() - same thing you did
.setOutputType(ImageSegmenter.ImageSegmenterOptions.OutputType.CONFIDENCE_MASK)
I changed the ResultBundle to accept a FloatBuffer
My returnSegmentationResult(...) function looks like:
val mpImage = result.segmentations().first()
imageSegmenterListener?.onResults(
ResultBundle(
ByteBufferExtractor.extract(mpImage).asFloatBuffer(),
mpImage.width,
mpImage.height,
inferenceTime
)
)
I commented a lot of stuff out (label colors, the adapter) to just make it work in a not-the-most-pretty way
In the OverlayView I changed setResults to the following (again, some artifacts from the existing sample - I didn't exactly clean anything up):
fun setResults(
byteBuffer: FloatBuffer,
outputWidth: Int,
outputHeight: Int
) {
val colorLabel = HashSet<Pair<String, Int>>()
// Create the mask bitmap with colors and the set of detected labels.
val pixels = IntArray(byteBuffer.capacity())
for (i in pixels.indices) {
val confidence = byteBuffer.get()
val color =
Color.argb(255 * if( (1.0f - confidence) >= 0.5f ) 0.0f else { confidence }, 0.0f, 0.0f, 128.0f)
pixels[i] = color
}
val image = Bitmap.createBitmap(
pixels,
outputWidth,
outputHeight,
Bitmap.Config.ARGB_8888
)
val scaleFactor = when (runningMode) {
RunningMode.IMAGE,
RunningMode.VIDEO -> {
min(width * 1f / outputWidth, height * 1f / outputHeight)
}
RunningMode.LIVE_STREAM -> {
// PreviewView is in FILL_START mode. So we need to scale up the
// landmarks to match with the size that the captured images will be
// displayed.
max(width * 1f / outputWidth, height * 1f / outputHeight)
}
}
val scaleWidth = (outputWidth * scaleFactor).toInt()
val scaleHeight = (outputHeight * scaleFactor).toInt()
scaleBitmap = Bitmap.createScaledBitmap(
image, scaleWidth, scaleHeight, false
)
invalidate()
listener?.onLabels(colorLabel.toList())
}
There's probably a few steps in there I missed (GalleryFragment had to change a placeholder resultsbundle, for example), but that should give you the general idea that led me to this:
https://photos.app.goo.gl/8g2Ea549NMhsbt7k7
from mediapipe-samples.
Hey, going to close this out since it's been a few days. Let me know if you have other questions though! :)
from mediapipe-samples.
havent try the solution you offer, will respond once I tried it
from mediapipe-samples.
Splendid!, it works, thanks
from mediapipe-samples.
Related Issues (20)
- LLM Inference Android Sample Issue: collectIndexed in sendMessage Causes Duplicate Collections on Multiple Invocations
- Can't Serve Files Locally in Python Due to CORS
- Image Segmentation Live Stream Mode for Python
- Crash when running Android text classification sample with MobileBERT. HOT 4
- Gesture Recognizer example with Raspberry Pi performance issues HOT 2
- Synchronize this proposal with the language steering group's amendment for the Swift 6 language mode, as [announced](https://forums.swift.org/t/amendment-se-0192-handling-future-enum-cases/68321) and [implemented for Swift 6.0](https://github.com/apple/swift/pull/73481).
- Can mediapipe-models be used for free for commercial purposes ? HOT 1
- RuntimeWarning: overflow encountered in cast v.numpy().astype("float16").tofile(output_bin_file) HOT 2
- Android LLM Inference Conversion guide for models fine tuned with keras nlp
- LLM Models other than Gemma , falcon, phi, stablelm HOT 1
- About face_detector performance issues in android HOT 3
- How to get LLM model performance? HOT 1
- Gemma-1.1-2B-it-cpu-int4.bin generating junk response in phone. HOT 4
- Gesture recognizer does not run on the emulator HOT 1
- LLM inference error when trying to run model locally, but works on Google's site HOT 1
- Native binaries are not compliant with Google Play 64-bit requirement HOT 1
- [Face Detection LandMark] Add a confidence score to the eye landmarks when wearing a black glasses
- Support for armeabi-v7a (32-bit ARM) the LLM Inference API
- The model is not a valid Flatbuffer buffer
- Unable to detect hand pose estimation for Android live app
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mediapipe-samples.