Hide PlaneRenderer when taking photos in Sceneform

I added a feature based on the Codelabs tutorial from Google (https://codelabs.developers.google.com/codelabs/sceneform-intro/index.html?index=..%2F..index#15) which allows users to take photos of AR objects that were added into the scene. The code works fine, however, I wish to hide the PlaneRenderer (the white dots that appear when ARCore detects a surface) in the photo taken by users.

In the onClickListener for the “Capture Photo” button, I tried setting PlaneRenderer to invisible before the takePhoto() is called. This hid the PlaneRenderer on screen, but not in the photo captured.

This is my onClickListener:

capturePhotoBtn.setOnClickListener(new View.OnClickListener(){
            @Override
            public void onClick(View v) {
                arFragment.getArSceneView().getPlaneRenderer().setVisible(false);
                for (TransformableNode vNode : videoNodeList){
                    if (vNode.isSelected()){
                        vNode.getTransformationSystem().selectNode(null);
                    }
                }
                takePhoto();
            }
        });

videoNodeList contains a list of transformableNodes, and is used to keep track of the objects added by users (as users can add more than 1 object into the scene). As the objects are transformableNodes, users can tap on them to resize/rotate, which shows a small circle underneath the selected object. So, the for-loop added is to de-select all transformableNodes when taking photos, to ensure that the small circle does not appear in the photo as well.

The takePhoto() method is from the CodeLabs tutorial, and is given as follows:

private void takePhoto() {

        final String filename = generateFilename();
        ArSceneView view = arFragment.getArSceneView();

        // Create a bitmap the size of the scene view.
        final Bitmap bitmap = Bitmap.createBitmap(view.getWidth(), view.getHeight(),
                Bitmap.Config.ARGB_8888);

        // Create a handler thread to offload the processing of the image.
        final HandlerThread handlerThread = new HandlerThread("PixelCopier");
        handlerThread.start();
        // Make the request to copy.
        PixelCopy.request(view, bitmap, (copyResult) -> {
            if (copyResult == PixelCopy.SUCCESS) {
                try {
                    File file = saveBitmapToDisk(bitmap, filename);
                    MediaScannerConnection.scanFile(this,
                            new String[] { file.toString() }, null,
                            new MediaScannerConnection.OnScanCompletedListener() {
                                public void onScanCompleted(String path, Uri uri) {
                                    Log.i("ExternalStorage", "Scanned " + path + ":");
                                    Log.i("ExternalStorage", "-> uri=" + uri);
                                }
                            });
                } catch (IOException e) {
                    Toast toast = Toast.makeText(ChromaKeyVideoActivity.this, e.toString(),
                            Toast.LENGTH_LONG);
                    toast.show();
                    return;
                }
                Snackbar snackbar = Snackbar.make(findViewById(android.R.id.content),
                        "Photo saved", Snackbar.LENGTH_LONG);
                snackbar.setAction("Open in Photos", v -> {
                    File photoFile = new File(filename);

                    Uri photoURI = FileProvider.getUriForFile(ChromaKeyVideoActivity.this,
                            ChromaKeyVideoActivity.this.getPackageName() + ".provider",
                            photoFile);
                    Intent intent = new Intent(Intent.ACTION_VIEW, photoURI);
                    intent.setDataAndType(photoURI, "image/*");
                    intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION);
                    startActivity(intent);

                });
                snackbar.show();
            } else {
                Toast toast = Toast.makeText(ChromaKeyVideoActivity.this,
                        "Failed to copyPixels: " + copyResult, Toast.LENGTH_LONG);
                toast.show();
            }
            handlerThread.quitSafely();
        }, new Handler(handlerThread.getLooper()));

    }

To give you a clearer picture, PlaneRenderer is hidden on the device screen when the “Capture Photo” button is tapped. This is what is seen immediately after the user taps on the “Capture Photo” button:

However, PlaneRenderer still appears in the photo taken. This is the resulting image that was taken:

which is not what I was looking for as I want to hide the PlaneRenderer in the photo (ie. photo taken should not have the white dots)

Users of this app add objects by selecting an object from the menu and tapping on the PlaneRenderer, so disabling the PlaneRenderer totally is not feasible. In addition, I have another video recording feature in the app that managed to successfully hide the PlaneRenderer in the recording by simply setting PlaneRenderer to invisible, so I am not sure why it doesn’t work when capturing photos.

Any help will be greatly appreciated! 🙂

Answer

Finally figured this out after countless of hours. Sharing my solution (may not be the best solution) in case anyone faces this same issue in the future.

I discovered that due to the handlerThread used, the takePhoto() method always happens before the PlaneRenderer was set to invisible whenever the button is tapped. So, I added a short delay to ensure that the reverse happens – ie. delay takePhoto() method for a short while such that the method will always happen after the planeRenderer is invisible.

Here is the code snippet:

capturePhotoBtn.setOnClickListener(new View.OnClickListener(){
            @Override
            public void onClick(View v) {
                arFragment.getArSceneView().getPlaneRenderer().setVisible(false);
                for (TransformableNode vNode : videoNodeList){
                    if (vNode.isSelected()){
                        vNode.getTransformationSystem().selectNode(null);
                    }
                }

                v.postDelayed(new Runnable() {
                    public void run() {
                        takePhoto();
                    }
                }, 80);
            }
        });

This method worked for me, but I am sure there are better solutions to solve this problem. Hope this helps someone with the same problem and feel free to contribute if you know of a better solution.

Advertisement

Answer